2024-09-16T12:46:46.171 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T12:46:46.188 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T12:46:46.284 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280 branch: wip-bharath9-testing-2024-09-13-1444-quincy description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_cache_writearound} email: sangadi@redhat.com first_in_suite: false job_id: '7907280' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-bharath9-testing-2024-09-13-1444-quincy ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(CACHE_POOL_NEAR_FULL\) - \(CACHE_POOL_NO_HIT_SET\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - mons down - mon down - out of quorum sha1: d75f23820806da622f561ee5ee7551015285993f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d75f23820806da622f561ee5ee7551015285993f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath9-testing-2024-09-13-1444-quincy sha1: d75f23820806da622f561ee5ee7551015285993f owner: scheduled_sangadi@teuthology priority: 130 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 9764 sha1: d75f23820806da622f561ee5ee7551015285993f sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-bharath9-testing-2024-09-13-1444-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d75f23820806da622f561ee5ee7551015285993f targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: null - exec: client.0: - sudo ceph osd pool create cache 4 - sudo ceph osd tier add rbd cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay rbd cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 60 - sudo ceph osd pool set cache target_max_objects 250 - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_10:44:09 tube: smithi user: sangadi verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T12:46:46.284 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa; will attempt to use it 2024-09-16T12:46:46.285 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa/tasks 2024-09-16T12:46:46.285 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T12:46:46.286 INFO:teuthology.task.internal:Checking packages... 2024-09-16T12:46:46.307 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd75f23820806da622f561ee5ee7551015285993f' 2024-09-16T12:46:46.307 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T12:46:46.307 INFO:teuthology.packaging:ref: None 2024-09-16T12:46:46.307 INFO:teuthology.packaging:tag: None 2024-09-16T12:46:46.307 INFO:teuthology.packaging:branch: wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-16T12:46:46.308 INFO:teuthology.packaging:sha1: d75f23820806da622f561ee5ee7551015285993f 2024-09-16T12:46:46.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-16T12:46:46.457 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1643-gd75f2382-1focal 2024-09-16T12:46:46.459 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T12:46:46.465 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T12:46:46.465 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T12:46:46.520 INFO:teuthology.task.internal:Saving configuration 2024-09-16T12:46:46.532 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T12:46:46.539 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T12:46:46.565 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 12:40:42.244995', 'locked_by': 'scheduled_sangadi@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-16T12:46:46.590 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 12:40:42.243498', 'locked_by': 'scheduled_sangadi@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-16T12:46:46.590 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T12:46:46.597 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-16T12:46:46.597 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-16T12:46:46.597 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T12:46:46.687 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1b1336fac0>, signals=[15]) 2024-09-16T12:46:46.687 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T12:46:46.693 INFO:teuthology.task.internal:Opening connections... 2024-09-16T12:46:46.693 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T12:46:46.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:46:46.784 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T12:46:46.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:46:46.871 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T12:46:46.878 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-09-16T12:46:46.884 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-09-16T12:46:46.884 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-09-16T12:46:46.934 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="Ubuntu" 2024-09-16T12:46:46.934 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-16T12:46:46.935 INFO:teuthology.orchestra.run.smithi042.stdout:ID=ubuntu 2024-09-16T12:46:46.935 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE=debian 2024-09-16T12:46:46.935 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-16T12:46:46.935 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="20.04" 2024-09-16T12:46:46.935 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T12:46:46.935 INFO:teuthology.orchestra.run.smithi042.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T12:46:46.935 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T12:46:46.935 INFO:teuthology.orchestra.run.smithi042.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T12:46:46.935 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_CODENAME=focal 2024-09-16T12:46:46.935 INFO:teuthology.orchestra.run.smithi042.stdout:UBUNTU_CODENAME=focal 2024-09-16T12:46:46.936 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-09-16T12:46:46.960 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-09-16T12:46:46.966 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-09-16T12:46:46.966 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-09-16T12:46:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu" 2024-09-16T12:46:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-16T12:46:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu 2024-09-16T12:46:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian 2024-09-16T12:46:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-16T12:46:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="20.04" 2024-09-16T12:46:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T12:46:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T12:46:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T12:46:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T12:46:47.014 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=focal 2024-09-16T12:46:47.014 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=focal 2024-09-16T12:46:47.014 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-09-16T12:46:47.035 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T12:46:47.043 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T12:46:47.049 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T12:46:47.049 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-09-16T12:46:47.051 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-09-16T12:46:47.057 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T12:46:47.062 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T12:46:47.062 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-09-16T12:46:47.064 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-09-16T12:46:47.085 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T12:46:47.139 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T12:46:47.154 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T12:46:47.154 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T12:46:47.154 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T12:46:47.155 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T12:46:47.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T12:46:47.160 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-09-16T12:46:47.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T12:46:47.161 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-09-16T12:46:47.167 INFO:teuthology.orchestra.run.smithi042.stdout:5.4.0-124-generic 2024-09-16T12:46:47.167 INFO:teuthology.task.kernel:Running kernel on smithi042: 5.4.0-124-generic 2024-09-16T12:46:47.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get clean 2024-09-16T12:46:47.168 INFO:teuthology.orchestra.run.smithi136.stdout:5.4.0-124-generic 2024-09-16T12:46:47.169 INFO:teuthology.task.kernel:Running kernel on smithi136: 5.4.0-124-generic 2024-09-16T12:46:47.169 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get clean 2024-09-16T12:46:47.269 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-09-16T12:46:47.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get update 2024-09-16T12:46:47.423 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T12:46:47.423 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-16T12:46:47.423 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-16T12:46:47.495 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-16T12:46:47.603 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T12:46:47.604 INFO:teuthology.orchestra.run.smithi042.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-16T12:46:47.633 INFO:teuthology.orchestra.run.smithi042.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-16T12:46:47.795 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-16T12:46:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-16T12:46:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-16T12:46:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-16T12:46:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-16T12:46:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-16T12:46:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-16T12:46:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-16T12:46:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T12:46:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-16T12:46:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-16T12:46:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-16T12:46:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-16T12:46:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-16T12:46:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-16T12:46:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-16T12:46:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-16T12:46:47.986 INFO:teuthology.orchestra.run.smithi042.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-16T12:46:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,566 kB] 2024-09-16T12:46:48.044 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-16T12:46:48.048 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-16T12:46:48.048 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-16T12:46:48.049 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,240 kB] 2024-09-16T12:46:48.071 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [454 kB] 2024-09-16T12:46:48.074 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T12:46:48.074 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-16T12:46:48.081 INFO:teuthology.orchestra.run.smithi042.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-16T12:46:48.083 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-16T12:46:48.088 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-16T12:46:48.090 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-16T12:46:48.091 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-16T12:46:48.091 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-16T12:46:48.091 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-16T12:46:48.097 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-16T12:46:48.131 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-16T12:46:48.131 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-16T12:46:48.133 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-16T12:46:48.133 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-16T12:46:48.134 INFO:teuthology.orchestra.run.smithi136.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-16T12:46:48.134 INFO:teuthology.orchestra.run.smithi136.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-16T12:46:48.227 INFO:teuthology.orchestra.run.smithi042.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,566 kB] 2024-09-16T12:46:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-16T12:46:48.601 INFO:teuthology.orchestra.run.smithi042.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-16T12:46:48.610 INFO:teuthology.orchestra.run.smithi042.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-16T12:46:48.610 INFO:teuthology.orchestra.run.smithi042.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-16T12:46:48.694 INFO:teuthology.orchestra.run.smithi042.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-16T12:46:48.699 INFO:teuthology.orchestra.run.smithi042.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-16T12:46:48.699 INFO:teuthology.orchestra.run.smithi042.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-16T12:46:48.707 INFO:teuthology.orchestra.run.smithi042.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T12:46:48.707 INFO:teuthology.orchestra.run.smithi042.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-16T12:46:48.710 INFO:teuthology.orchestra.run.smithi042.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-16T12:46:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-16T12:46:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-16T12:46:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,240 kB] 2024-09-16T12:46:48.758 INFO:teuthology.orchestra.run.smithi042.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-16T12:46:48.770 INFO:teuthology.orchestra.run.smithi042.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-16T12:46:48.772 INFO:teuthology.orchestra.run.smithi042.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-16T12:46:48.772 INFO:teuthology.orchestra.run.smithi042.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-16T12:46:48.772 INFO:teuthology.orchestra.run.smithi042.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-16T12:46:48.810 INFO:teuthology.orchestra.run.smithi042.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [454 kB] 2024-09-16T12:46:48.817 INFO:teuthology.orchestra.run.smithi042.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T12:46:48.817 INFO:teuthology.orchestra.run.smithi042.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-16T12:46:48.847 INFO:teuthology.orchestra.run.smithi042.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-16T12:46:48.847 INFO:teuthology.orchestra.run.smithi042.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-16T12:46:48.873 INFO:teuthology.orchestra.run.smithi042.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-16T12:46:48.885 INFO:teuthology.orchestra.run.smithi042.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-16T12:46:48.891 INFO:teuthology.orchestra.run.smithi042.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-16T12:46:48.891 INFO:teuthology.orchestra.run.smithi042.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-16T12:46:48.892 INFO:teuthology.orchestra.run.smithi042.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-16T12:46:48.892 INFO:teuthology.orchestra.run.smithi042.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-16T12:46:48.956 INFO:teuthology.orchestra.run.smithi042.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-16T12:46:48.956 INFO:teuthology.orchestra.run.smithi042.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-16T12:46:48.957 INFO:teuthology.orchestra.run.smithi042.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-16T12:46:48.958 INFO:teuthology.orchestra.run.smithi042.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-16T12:46:49.038 INFO:teuthology.orchestra.run.smithi042.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-16T12:46:49.038 INFO:teuthology.orchestra.run.smithi042.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-16T12:46:49.039 INFO:teuthology.orchestra.run.smithi042.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-16T12:46:52.195 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 21.9 MB in 3s (7,300 kB/s) 2024-09-16T12:46:52.944 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 21.9 MB in 4s (5,885 kB/s) 2024-09-16T12:46:53.759 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-16T12:46:53.786 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T12:46:53.870 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-16T12:46:54.065 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-09-16T12:46:54.065 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-09-16T12:46:54.263 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2024-09-16T12:46:54.263 INFO:teuthology.orchestra.run.smithi136.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T12:46:54.264 INFO:teuthology.orchestra.run.smithi136.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-16T12:46:54.264 INFO:teuthology.orchestra.run.smithi136.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-16T12:46:54.265 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2024-09-16T12:46:54.265 INFO:teuthology.orchestra.run.smithi136.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-16T12:46:54.265 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2024-09-16T12:46:54.265 INFO:teuthology.orchestra.run.smithi136.stdout: thermald 2024-09-16T12:46:54.324 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2024-09-16T12:46:54.324 INFO:teuthology.orchestra.run.smithi136.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T12:46:54.325 INFO:teuthology.orchestra.run.smithi136.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-16T12:46:54.325 INFO:teuthology.orchestra.run.smithi136.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-16T12:46:54.327 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages will be upgraded: 2024-09-16T12:46:54.327 INFO:teuthology.orchestra.run.smithi136.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T12:46:54.510 INFO:teuthology.orchestra.run.smithi136.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T12:46:54.511 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 77.2 MB of archives. 2024-09-16T12:46:54.511 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-16T12:46:54.511 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-16T12:46:54.640 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-16T12:46:54.667 DEBUG:teuthology.orchestra.run.smithi042:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T12:46:54.745 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-16T12:46:54.970 INFO:teuthology.orchestra.run.smithi042.stdout:Building dependency tree... 2024-09-16T12:46:54.970 INFO:teuthology.orchestra.run.smithi042.stdout:Reading state information... 2024-09-16T12:46:55.181 INFO:teuthology.orchestra.run.smithi042.stdout:The following additional packages will be installed: 2024-09-16T12:46:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T12:46:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-16T12:46:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-16T12:46:55.182 INFO:teuthology.orchestra.run.smithi042.stdout:Suggested packages: 2024-09-16T12:46:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-16T12:46:55.182 INFO:teuthology.orchestra.run.smithi042.stdout:Recommended packages: 2024-09-16T12:46:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: thermald 2024-09-16T12:46:55.246 INFO:teuthology.orchestra.run.smithi042.stdout:The following NEW packages will be installed: 2024-09-16T12:46:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T12:46:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-16T12:46:55.246 INFO:teuthology.orchestra.run.smithi042.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-16T12:46:55.248 INFO:teuthology.orchestra.run.smithi042.stdout:The following packages will be upgraded: 2024-09-16T12:46:55.248 INFO:teuthology.orchestra.run.smithi042.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T12:46:55.303 INFO:teuthology.orchestra.run.smithi042.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T12:46:55.303 INFO:teuthology.orchestra.run.smithi042.stdout:Need to get 77.2 MB of archives. 2024-09-16T12:46:55.303 INFO:teuthology.orchestra.run.smithi042.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-16T12:46:55.303 INFO:teuthology.orchestra.run.smithi042.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-16T12:46:55.607 INFO:teuthology.orchestra.run.smithi042.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-16T12:46:55.678 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-16T12:46:55.700 INFO:teuthology.orchestra.run.smithi042.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-16T12:46:55.948 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-16T12:46:56.167 INFO:teuthology.orchestra.run.smithi042.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-16T12:46:56.167 INFO:teuthology.orchestra.run.smithi042.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-16T12:46:56.167 INFO:teuthology.orchestra.run.smithi042.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-16T12:46:56.261 INFO:teuthology.orchestra.run.smithi042.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-16T12:46:56.272 INFO:teuthology.orchestra.run.smithi042.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-16T12:46:56.980 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 77.2 MB in 1s (75.9 MB/s) 2024-09-16T12:46:57.172 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-16T12:46:57.268 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-16T12:46:57.268 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-16T12:46:57.268 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-16T12:46:57.551 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-16T12:46:57.589 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-16T12:46:58.316 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 77.2 MB in 3s (23.7 MB/s) 2024-09-16T12:46:58.500 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-16T12:46:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-16T12:46:58.857 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T12:46:58.969 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:00.220 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-16T12:47:00.225 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T12:47:00.329 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:00.895 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-16T12:47:00.911 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T12:47:00.998 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:02.073 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-16T12:47:02.086 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T12:47:02.114 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:02.345 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-16T12:47:02.360 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T12:47:02.457 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:03.540 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-16T12:47:03.556 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T12:47:03.615 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:06.890 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T12:47:07.013 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T12:47:07.365 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T12:47:07.483 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T12:47:07.828 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-16T12:47:07.842 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-16T12:47:07.886 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T12:47:08.404 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T12:47:08.562 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T12:47:09.280 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T12:47:09.432 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T12:47:09.769 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-16T12:47:09.784 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-16T12:47:09.827 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T12:47:13.230 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-16T12:47:13.253 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T12:47:13.296 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:15.365 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-16T12:47:15.389 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T12:47:15.431 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:16.151 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T12:47:16.272 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T12:47:16.623 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:17.878 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:18.159 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T12:47:18.282 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T12:47:18.643 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:19.077 INFO:teuthology.orchestra.run.smithi042.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-16T12:47:19.077 INFO:teuthology.orchestra.run.smithi042.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-16T12:47:19.375 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:20.503 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:21.382 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T12:47:21.508 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-16T12:47:21.634 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:21.646 INFO:teuthology.orchestra.run.smithi136.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-16T12:47:21.646 INFO:teuthology.orchestra.run.smithi136.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-16T12:47:21.780 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:21.785 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-16T12:47:21.911 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-16T12:47:22.038 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:22.107 INFO:teuthology.orchestra.run.smithi042.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T12:47:22.108 INFO:teuthology.orchestra.run.smithi042.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-16T12:47:22.998 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T12:47:23.124 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-16T12:47:23.271 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:23.401 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-16T12:47:23.527 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-16T12:47:23.654 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T12:47:23.723 INFO:teuthology.orchestra.run.smithi136.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T12:47:23.723 INFO:teuthology.orchestra.run.smithi136.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-16T12:47:44.337 INFO:teuthology.orchestra.run.smithi042.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T12:47:44.337 INFO:teuthology.orchestra.run.smithi042.stdout:Sourcing file `/etc/default/grub' 2024-09-16T12:47:44.346 INFO:teuthology.orchestra.run.smithi042.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T12:47:44.357 INFO:teuthology.orchestra.run.smithi042.stdout:Generating grub configuration file ... 2024-09-16T12:47:44.709 INFO:teuthology.orchestra.run.smithi042.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T12:47:44.726 INFO:teuthology.orchestra.run.smithi042.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T12:47:45.129 INFO:teuthology.orchestra.run.smithi042.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T12:47:45.141 INFO:teuthology.orchestra.run.smithi042.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T12:47:45.521 INFO:teuthology.orchestra.run.smithi136.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T12:47:45.521 INFO:teuthology.orchestra.run.smithi136.stdout:Sourcing file `/etc/default/grub' 2024-09-16T12:47:45.526 INFO:teuthology.orchestra.run.smithi136.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T12:47:45.548 INFO:teuthology.orchestra.run.smithi136.stdout:Generating grub configuration file ... 2024-09-16T12:47:45.860 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T12:47:45.894 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T12:47:46.276 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T12:47:46.287 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T12:47:47.882 INFO:teuthology.orchestra.run.smithi042.stdout:done 2024-09-16T12:47:48.328 DEBUG:teuthology.orchestra.run.smithi042:> dpkg -s linux-image-generic 2024-09-16T12:47:48.372 INFO:teuthology.orchestra.run.smithi042.stdout:Package: linux-image-generic 2024-09-16T12:47:48.372 INFO:teuthology.orchestra.run.smithi042.stdout:Status: install ok installed 2024-09-16T12:47:48.372 INFO:teuthology.orchestra.run.smithi042.stdout:Priority: optional 2024-09-16T12:47:48.372 INFO:teuthology.orchestra.run.smithi042.stdout:Section: kernel 2024-09-16T12:47:48.372 INFO:teuthology.orchestra.run.smithi042.stdout:Installed-Size: 20 2024-09-16T12:47:48.372 INFO:teuthology.orchestra.run.smithi042.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T12:47:48.373 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: amd64 2024-09-16T12:47:48.373 INFO:teuthology.orchestra.run.smithi042.stdout:Source: linux-meta 2024-09-16T12:47:48.373 INFO:teuthology.orchestra.run.smithi042.stdout:Version: 5.4.0.195.193 2024-09-16T12:47:48.373 INFO:teuthology.orchestra.run.smithi042.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T12:47:48.373 INFO:teuthology.orchestra.run.smithi042.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T12:47:48.373 INFO:teuthology.orchestra.run.smithi042.stdout:Recommends: thermald 2024-09-16T12:47:48.373 INFO:teuthology.orchestra.run.smithi042.stdout:Description: Generic Linux kernel image 2024-09-16T12:47:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: This package will always depend on the latest generic kernel image 2024-09-16T12:47:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: available. 2024-09-16T12:47:48.374 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-16T12:47:48.374 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T12:47:48.374 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T12:47:48.374 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T12:47:48.374 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T12:47:48.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get clean 2024-09-16T12:47:48.383 INFO:teuthology.orchestra.run.smithi136.stdout:done 2024-09-16T12:47:48.454 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get update 2024-09-16T12:47:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T12:47:48.704 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T12:47:48.785 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T12:47:48.814 DEBUG:teuthology.orchestra.run.smithi136:> dpkg -s linux-image-generic 2024-09-16T12:47:48.867 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T12:47:48.868 INFO:teuthology.orchestra.run.smithi136.stdout:Package: linux-image-generic 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Status: install ok installed 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Priority: optional 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Section: kernel 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Installed-Size: 20 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: amd64 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Source: linux-meta 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Version: 5.4.0.195.193 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T12:47:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Recommends: thermald 2024-09-16T12:47:48.870 INFO:teuthology.orchestra.run.smithi136.stdout:Description: Generic Linux kernel image 2024-09-16T12:47:48.870 INFO:teuthology.orchestra.run.smithi136.stdout: This package will always depend on the latest generic kernel image 2024-09-16T12:47:48.870 INFO:teuthology.orchestra.run.smithi136.stdout: available. 2024-09-16T12:47:48.870 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-16T12:47:48.870 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T12:47:48.870 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T12:47:48.870 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T12:47:48.870 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T12:47:48.870 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get clean 2024-09-16T12:47:48.961 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-09-16T12:47:49.092 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T12:47:49.210 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T12:47:49.291 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T12:47:49.373 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T12:47:50.511 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-16T12:47:50.538 DEBUG:teuthology.orchestra.run.smithi042:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T12:47:50.608 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-16T12:47:50.833 INFO:teuthology.orchestra.run.smithi042.stdout:Building dependency tree... 2024-09-16T12:47:50.834 INFO:teuthology.orchestra.run.smithi042.stdout:Reading state information... 2024-09-16T12:47:50.933 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-16T12:47:50.960 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T12:47:51.031 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-16T12:47:51.086 INFO:teuthology.orchestra.run.smithi042.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-16T12:47:51.086 INFO:teuthology.orchestra.run.smithi042.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T12:47:51.088 DEBUG:teuthology.orchestra.run.smithi042:> dpkg -s linux-image-generic 2024-09-16T12:47:51.111 INFO:teuthology.orchestra.run.smithi042.stdout:Package: linux-image-generic 2024-09-16T12:47:51.111 INFO:teuthology.orchestra.run.smithi042.stdout:Status: install ok installed 2024-09-16T12:47:51.111 INFO:teuthology.orchestra.run.smithi042.stdout:Priority: optional 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout:Section: kernel 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout:Installed-Size: 20 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: amd64 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout:Source: linux-meta 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout:Version: 5.4.0.195.193 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout:Recommends: thermald 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout:Description: Generic Linux kernel image 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: This package will always depend on the latest generic kernel image 2024-09-16T12:47:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: available. 2024-09-16T12:47:51.113 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-16T12:47:51.118 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.DB7lj12qbC 2024-09-16T12:47:51.119 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub/grub.cfg /tmp/tmp.DB7lj12qbC 2024-09-16T12:47:51.151 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.DB7lj12qbC 2024-09-16T12:47:51.224 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-09-16T12:47:51.224 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-09-16T12:47:51.245 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.DB7lj12qbC is 9KB 2024-09-16T12:47:51.258 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.DB7lj12qbC 2024-09-16T12:47:51.264 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T12:47:51.277 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:47:51.277 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T12:47:51.277 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T12:47:51.354 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-16T12:47:51.355 DEBUG:teuthology.orchestra.run.smithi042:> sudo update-grub 2024-09-16T12:47:51.453 INFO:teuthology.orchestra.run.smithi136.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-16T12:47:51.453 INFO:teuthology.orchestra.run.smithi136.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T12:47:51.455 DEBUG:teuthology.orchestra.run.smithi136:> dpkg -s linux-image-generic 2024-09-16T12:47:51.475 INFO:teuthology.orchestra.run.smithi136.stdout:Package: linux-image-generic 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Status: install ok installed 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Priority: optional 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Section: kernel 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Installed-Size: 20 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: amd64 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Source: linux-meta 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Version: 5.4.0.195.193 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Recommends: thermald 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout:Description: Generic Linux kernel image 2024-09-16T12:47:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: This package will always depend on the latest generic kernel image 2024-09-16T12:47:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: available. 2024-09-16T12:47:51.477 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-09-16T12:47:51.483 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.mgf2LqpiaL 2024-09-16T12:47:51.483 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub/grub.cfg /tmp/tmp.mgf2LqpiaL 2024-09-16T12:47:51.502 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.mgf2LqpiaL 2024-09-16T12:47:51.535 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.mgf2LqpiaL is 9KB 2024-09-16T12:47:51.549 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.mgf2LqpiaL 2024-09-16T12:47:51.556 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T12:47:51.612 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:47:51.612 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T12:47:51.612 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T12:47:51.649 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-16T12:47:51.649 DEBUG:teuthology.orchestra.run.smithi136:> sudo update-grub 2024-09-16T12:47:52.826 INFO:teuthology.orchestra.run.smithi042.stderr:Sourcing file `/etc/default/grub' 2024-09-16T12:47:52.831 INFO:teuthology.orchestra.run.smithi042.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T12:47:52.845 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-09-16T12:47:53.075 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T12:47:53.086 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T12:47:53.093 INFO:teuthology.orchestra.run.smithi136.stderr:Sourcing file `/etc/default/grub' 2024-09-16T12:47:53.104 INFO:teuthology.orchestra.run.smithi136.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T12:47:53.112 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2024-09-16T12:47:53.352 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T12:47:53.364 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T12:47:53.451 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T12:47:53.463 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T12:47:53.701 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T12:47:53.712 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T12:47:54.688 INFO:teuthology.orchestra.run.smithi136.stderr:done 2024-09-16T12:47:54.690 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2024-09-16T12:47:55.081 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-09-16T12:47:55.082 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-09-16T12:48:24.762 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T12:48:24.762 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-16T12:48:24.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:48:25.085 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T12:48:25.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T12:48:25.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:48:43.325 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-09-16T12:48:43.581 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-09-16T12:48:52.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-16T12:48:52.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:48:52.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T12:48:52.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:48:55.389 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-09-16T12:49:07.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-16T12:49:07.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:49:25.821 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-09-16T12:49:40.836 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-16T12:49:40.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:49:41.531 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-09-16T12:49:42.083 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-16T12:49:42.083 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-195-generic"... 2024-09-16T12:49:42.083 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-09-16T12:49:42.089 INFO:teuthology.orchestra.run.smithi136.stdout:5.4.0-195-generic 2024-09-16T12:49:42.089 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-16T12:49:42.090 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T12:49:42.090 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T12:49:43.091 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T12:49:43.091 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T12:49:43.193 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2024-09-16T12:49:43.217 DEBUG:teuthology.parallel:result is None 2024-09-16T12:49:52.585 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T12:50:04.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T12:50:04.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:50:04.835 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-09-16T12:50:05.673 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T12:50:05.673 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-16T12:50:05.673 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-09-16T12:50:05.680 INFO:teuthology.orchestra.run.smithi042.stdout:5.4.0-195-generic 2024-09-16T12:50:05.680 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-16T12:50:05.680 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T12:50:05.680 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T12:50:06.682 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T12:50:06.682 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T12:50:06.808 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-09-16T12:50:06.841 DEBUG:teuthology.parallel:result is None 2024-09-16T12:50:06.841 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T12:50:06.848 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T12:50:06.848 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T12:50:06.850 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T12:50:06.857 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T12:50:06.925 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T12:50:06.935 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T12:50:06.935 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T12:50:06.937 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T12:50:06.957 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T12:50:06.964 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T12:50:06.964 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T12:50:06.966 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T12:50:06.982 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T12:50:06.996 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T12:50:07.016 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T12:50:07.026 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T12:50:07.028 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T12:50:07.035 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T12:50:07.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T12:50:07.040 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T12:50:07.054 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T12:50:07.063 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T12:50:07.063 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T12:50:07.065 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T12:50:07.101 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T12:50:07.117 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T12:50:07.123 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:50:07.123 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T12:50:07.152 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T12:50:07.159 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T12:50:07.166 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:50:07.166 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T12:50:07.192 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-09-16T12:50:07.194 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-09-16T12:50:07.288 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T12:50:07.295 INFO:teuthology.task.internal:Starting timer... 2024-09-16T12:50:07.295 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T12:50:07.311 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T12:50:07.318 DEBUG:teuthology.task.selinux:Excluding smithi042: OS 'ubuntu' does not support SELinux 2024-09-16T12:50:07.318 DEBUG:teuthology.task.selinux:Excluding smithi136: OS 'ubuntu' does not support SELinux 2024-09-16T12:50:07.318 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T12:50:07.318 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-16T12:50:07.318 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T12:50:07.318 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T12:50:07.325 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T12:50:07.337 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T12:50:07.338 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T12:55:44.693 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2024-09-16T12:55:44.694 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T12:55:44.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:55:44.794 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-09-16T12:55:44.922 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T12:55:44.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-16T12:55:44.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T12:55:45.021 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-09-16T12:55:45.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-16T12:55:45.146 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T12:55:45.158 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T12:55:45.158 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T12:55:45.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T12:55:45.161 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T12:55:45.161 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T12:55:45.195 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-16T12:55:45.195 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: Command line: ntpd -gq 2024-09-16T12:55:45.197 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: proto: precision = 0.120 usec (-23) 2024-09-16T12:55:45.198 INFO:teuthology.orchestra.run.smithi042.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T12:55:45.198 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T12:55:45.199 INFO:teuthology.orchestra.run.smithi042.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T12:55:45.200 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T12:55:45.200 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T12:55:45.200 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T12:55:45.200 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T12:55:45.200 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: Listen normally on 3 ens1f0 172.21.15.42:123 2024-09-16T12:55:45.200 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: Listen normally on 4 lo [::1]:123 2024-09-16T12:55:45.200 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1440%4]:123 2024-09-16T12:55:45.200 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:45 ntpd[13708]: Listening on routing socket on fd #22 for interface updates 2024-09-16T12:55:45.222 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-16T12:55:45.222 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: Command line: ntpd -gq 2024-09-16T12:55:45.223 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: proto: precision = 0.087 usec (-23) 2024-09-16T12:55:45.224 INFO:teuthology.orchestra.run.smithi136.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T12:55:45.224 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T12:55:45.224 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T12:55:45.224 INFO:teuthology.orchestra.run.smithi136.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T12:55:45.225 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T12:55:45.225 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T12:55:45.225 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T12:55:45.226 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: Listen normally on 3 enp3s0f1 172.21.15.136:123 2024-09-16T12:55:45.226 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: Listen normally on 4 lo [::1]:123 2024-09-16T12:55:45.226 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e43%5]:123 2024-09-16T12:55:45.226 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:45 ntpd[13756]: Listening on routing socket on fd #22 for interface updates 2024-09-16T12:55:46.196 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:46 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:46.196 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:46 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:46.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:46 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:46.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:46 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:46.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:46 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:46.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:46 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:46.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:46 ntpd[13756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:46.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:46 ntpd[13756]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:47.222 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:47 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:47.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:47 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:48.196 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:48 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:48.197 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:48 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:48.197 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:48 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:48.197 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:48 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:48.197 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:48 ntpd[13708]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:48.197 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:48 ntpd[13708]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:48.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:48 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:48.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:48 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:48.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:48 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:48.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:48 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:48.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:48 ntpd[13756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:48.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:48 ntpd[13756]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:49.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:49 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:49.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:49 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:49.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:49 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:49.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:49 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:49.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:49 ntpd[13756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:49.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:49 ntpd[13756]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:50.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:50.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:50.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:50.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:50.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:50.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:50.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:50 ntpd[13708]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:50.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:50.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:50.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:50.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:50.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:50.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:50.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:50.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:50.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:50.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:50.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:50.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:50 ntpd[13756]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:51.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:51 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:51.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:51 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:51.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:51 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:51.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:51 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:51.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:51 ntpd[13756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:51.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:51 ntpd[13756]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:52.197 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:52.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:52.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:52.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:52.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:52.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:52.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:52 ntpd[13708]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:52.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:52.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:52.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:52.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:52.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:52.223 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:52.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:52.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:52.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:52.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:52.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:52.224 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:52.224 INFO:teuthology.orchestra.run.smithi136.stdout:16 Sep 12:55:52 ntpd[13756]: ntpd: time slew +0.001180 s 2024-09-16T12:55:52.224 INFO:teuthology.orchestra.run.smithi136.stdout:ntpd: time slew +0.001180s 2024-09-16T12:55:52.225 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T12:55:52.225 INFO:teuthology.orchestra.run.smithi136.stderr:16 Sep 12:55:52 ntpd[13756]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T12:55:52.299 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T12:55:52.299 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2024-09-16T12:55:52.299 INFO:teuthology.orchestra.run.smithi136.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T12:55:52.299 INFO:teuthology.orchestra.run.smithi136.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T12:55:52.299 INFO:teuthology.orchestra.run.smithi136.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T12:55:52.299 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:54.198 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T12:55:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T12:55:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T12:55:54.199 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T12:55:54.199 INFO:teuthology.orchestra.run.smithi042.stdout:16 Sep 12:55:54 ntpd[13708]: ntpd: time slew +0.003679 s 2024-09-16T12:55:54.199 INFO:teuthology.orchestra.run.smithi042.stdout:ntpd: time slew +0.003679s 2024-09-16T12:55:54.200 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T12:55:54.200 INFO:teuthology.orchestra.run.smithi042.stderr:16 Sep 12:55:54 ntpd[13708]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T12:55:54.267 INFO:teuthology.orchestra.run.smithi042.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T12:55:54.267 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================== 2024-09-16T12:55:54.267 INFO:teuthology.orchestra.run.smithi042.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T12:55:54.267 INFO:teuthology.orchestra.run.smithi042.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T12:55:54.267 INFO:teuthology.orchestra.run.smithi042.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T12:55:54.267 INFO:teuthology.orchestra.run.smithi042.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T12:55:54.268 INFO:teuthology.run_tasks:Running task install... 2024-09-16T12:55:54.276 DEBUG:teuthology.task.install:project ceph 2024-09-16T12:55:54.276 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-16T12:55:54.276 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'} 2024-09-16T12:55:54.276 INFO:teuthology.task.install:Using flavor: default 2024-09-16T12:55:54.281 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T12:55:54.282 INFO:teuthology.task.install:extra packages: [] 2024-09-16T12:55:54.282 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-key list | grep Ceph 2024-09-16T12:55:54.282 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-key list | grep Ceph 2024-09-16T12:55:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-16T12:55:54.385 INFO:teuthology.orchestra.run.smithi136.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-16T12:55:54.444 INFO:teuthology.orchestra.run.smithi042.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T12:55:54.444 INFO:teuthology.orchestra.run.smithi042.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T12:55:54.445 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T12:55:54.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T12:55:54.450 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T12:55:54.450 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T12:55:54.450 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T12:55:54.451 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T12:55:54.606 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-16T12:55:54.606 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-16T12:55:54.609 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-16T12:55:54.609 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-16T12:55:54.713 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:55:54.713 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T12:55:54.722 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:55:54.722 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T12:55:54.729 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-09-16T12:55:54.738 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get update 2024-09-16T12:55:54.881 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T12:55:54.915 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T12:55:54.916 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T12:55:54.921 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T12:55:54.932 INFO:teuthology.orchestra.run.smithi042.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-16T12:55:54.942 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T12:55:54.954 INFO:teuthology.orchestra.run.smithi042.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-16T12:55:54.967 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-16T12:55:54.976 INFO:teuthology.orchestra.run.smithi042.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-16T12:55:54.988 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-16T12:55:54.996 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T12:55:54.998 INFO:teuthology.orchestra.run.smithi042.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-16T12:55:55.013 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-16T12:55:55.032 INFO:teuthology.orchestra.run.smithi042.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-16T12:55:55.034 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-16T12:55:55.067 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-16T12:55:55.078 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T12:55:55.160 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T12:55:55.562 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 28.5 kB in 1s (43.5 kB/s) 2024-09-16T12:55:55.565 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 28.5 kB in 1s (45.1 kB/s) 2024-09-16T12:55:56.731 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-16T12:55:56.758 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-16T12:55:56.770 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-16T12:55:56.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-16T12:55:56.832 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-16T12:55:56.868 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-16T12:55:57.049 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-09-16T12:55:57.049 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-09-16T12:55:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:Building dependency tree... 2024-09-16T12:55:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:Reading state information... 2024-09-16T12:55:57.246 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T12:55:57.247 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-16T12:55:57.247 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T12:55:57.248 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2024-09-16T12:55:57.248 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T12:55:57.248 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-16T12:55:57.248 INFO:teuthology.orchestra.run.smithi136.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-16T12:55:57.248 INFO:teuthology.orchestra.run.smithi136.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-16T12:55:57.248 INFO:teuthology.orchestra.run.smithi136.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-16T12:55:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-16T12:55:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-16T12:55:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-16T12:55:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-16T12:55:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-16T12:55:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-16T12:55:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-16T12:55:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-16T12:55:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T12:55:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-16T12:55:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T12:55:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-16T12:55:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T12:55:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-16T12:55:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-16T12:55:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-16T12:55:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: socat xmlstarlet 2024-09-16T12:55:57.250 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2024-09-16T12:55:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: btrfs-tools 2024-09-16T12:55:57.272 INFO:teuthology.orchestra.run.smithi042.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T12:55:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-16T12:55:57.274 INFO:teuthology.orchestra.run.smithi042.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T12:55:57.274 INFO:teuthology.orchestra.run.smithi042.stdout:The following additional packages will be installed: 2024-09-16T12:55:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T12:55:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-16T12:55:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-16T12:55:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-16T12:55:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-16T12:55:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-16T12:55:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-16T12:55:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-16T12:55:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-16T12:55:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-16T12:55:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-16T12:55:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-16T12:55:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-16T12:55:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T12:55:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-16T12:55:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T12:55:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-16T12:55:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T12:55:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-16T12:55:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-16T12:55:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-16T12:55:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: socat xmlstarlet 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout:Suggested packages: 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout:Recommended packages: 2024-09-16T12:55:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: btrfs-tools 2024-09-16T12:55:57.369 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2024-09-16T12:55:57.369 INFO:teuthology.orchestra.run.smithi136.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T12:55:57.370 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T12:55:57.370 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-16T12:55:57.370 INFO:teuthology.orchestra.run.smithi136.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-16T12:55:57.370 INFO:teuthology.orchestra.run.smithi136.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-16T12:55:57.370 INFO:teuthology.orchestra.run.smithi136.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-16T12:55:57.370 INFO:teuthology.orchestra.run.smithi136.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-16T12:55:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-16T12:55:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-16T12:55:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-16T12:55:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-16T12:55:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-16T12:55:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-16T12:55:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-16T12:55:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-16T12:55:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-16T12:55:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T12:55:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-16T12:55:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-16T12:55:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-16T12:55:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-16T12:55:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-16T12:55:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-16T12:55:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-16T12:55:57.372 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages will be upgraded: 2024-09-16T12:55:57.373 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 librbd1 2024-09-16T12:55:57.400 INFO:teuthology.orchestra.run.smithi042.stdout:The following NEW packages will be installed: 2024-09-16T12:55:57.400 INFO:teuthology.orchestra.run.smithi042.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T12:55:57.400 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T12:55:57.400 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-16T12:55:57.402 INFO:teuthology.orchestra.run.smithi042.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-16T12:55:57.402 INFO:teuthology.orchestra.run.smithi042.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-16T12:55:57.402 INFO:teuthology.orchestra.run.smithi042.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-16T12:55:57.403 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-16T12:55:57.404 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-16T12:55:57.404 INFO:teuthology.orchestra.run.smithi042.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-16T12:55:57.404 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-16T12:55:57.404 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-16T12:55:57.404 INFO:teuthology.orchestra.run.smithi042.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-16T12:55:57.404 INFO:teuthology.orchestra.run.smithi042.stdout:The following packages will be upgraded: 2024-09-16T12:55:57.404 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 librbd1 2024-09-16T12:55:57.469 INFO:teuthology.orchestra.run.smithi042.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-16T12:55:57.469 INFO:teuthology.orchestra.run.smithi042.stdout:Need to get 201 MB of archives. 2024-09-16T12:55:57.469 INFO:teuthology.orchestra.run.smithi042.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-16T12:55:57.469 INFO:teuthology.orchestra.run.smithi042.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-16T12:55:57.520 INFO:teuthology.orchestra.run.smithi136.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-16T12:55:57.520 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 201 MB of archives. 2024-09-16T12:55:57.520 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-16T12:55:57.520 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-16T12:55:57.552 INFO:teuthology.orchestra.run.smithi042.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-16T12:55:57.553 INFO:teuthology.orchestra.run.smithi042.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-16T12:55:57.555 INFO:teuthology.orchestra.run.smithi042.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-16T12:55:57.564 INFO:teuthology.orchestra.run.smithi042.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-16T12:55:57.564 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-16T12:55:57.567 INFO:teuthology.orchestra.run.smithi042.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-16T12:55:57.594 INFO:teuthology.orchestra.run.smithi042.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-16T12:55:57.595 INFO:teuthology.orchestra.run.smithi042.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-16T12:55:57.601 INFO:teuthology.orchestra.run.smithi042.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-16T12:55:57.636 INFO:teuthology.orchestra.run.smithi042.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-16T12:55:57.638 INFO:teuthology.orchestra.run.smithi042.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-16T12:55:57.644 INFO:teuthology.orchestra.run.smithi042.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-16T12:55:57.646 INFO:teuthology.orchestra.run.smithi042.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-16T12:55:57.646 INFO:teuthology.orchestra.run.smithi042.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-16T12:55:57.648 INFO:teuthology.orchestra.run.smithi042.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-16T12:55:57.648 INFO:teuthology.orchestra.run.smithi042.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-16T12:55:57.649 INFO:teuthology.orchestra.run.smithi042.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-16T12:55:57.649 INFO:teuthology.orchestra.run.smithi042.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-16T12:55:57.654 INFO:teuthology.orchestra.run.smithi042.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-16T12:55:57.668 INFO:teuthology.orchestra.run.smithi042.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-16T12:55:57.669 INFO:teuthology.orchestra.run.smithi042.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-16T12:55:57.669 INFO:teuthology.orchestra.run.smithi042.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-16T12:55:57.670 INFO:teuthology.orchestra.run.smithi042.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-16T12:55:57.671 INFO:teuthology.orchestra.run.smithi042.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T12:55:57.671 INFO:teuthology.orchestra.run.smithi042.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-16T12:55:57.672 INFO:teuthology.orchestra.run.smithi042.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-16T12:55:57.673 INFO:teuthology.orchestra.run.smithi042.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-16T12:55:57.674 INFO:teuthology.orchestra.run.smithi042.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-16T12:55:57.688 INFO:teuthology.orchestra.run.smithi042.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-16T12:55:57.690 INFO:teuthology.orchestra.run.smithi042.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-16T12:55:57.691 INFO:teuthology.orchestra.run.smithi042.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-16T12:55:57.705 INFO:teuthology.orchestra.run.smithi042.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-16T12:55:57.706 INFO:teuthology.orchestra.run.smithi042.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-16T12:55:57.706 INFO:teuthology.orchestra.run.smithi042.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-16T12:55:57.706 INFO:teuthology.orchestra.run.smithi042.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-16T12:55:57.707 INFO:teuthology.orchestra.run.smithi042.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-16T12:55:57.709 INFO:teuthology.orchestra.run.smithi042.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-16T12:55:57.710 INFO:teuthology.orchestra.run.smithi042.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T12:55:57.723 INFO:teuthology.orchestra.run.smithi042.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-16T12:55:57.724 INFO:teuthology.orchestra.run.smithi042.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-16T12:55:57.726 INFO:teuthology.orchestra.run.smithi042.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-16T12:55:57.740 INFO:teuthology.orchestra.run.smithi042.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-16T12:55:57.740 INFO:teuthology.orchestra.run.smithi042.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-16T12:55:57.741 INFO:teuthology.orchestra.run.smithi042.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-16T12:55:57.828 INFO:teuthology.orchestra.run.smithi042.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-16T12:55:57.843 INFO:teuthology.orchestra.run.smithi042.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-16T12:55:57.845 INFO:teuthology.orchestra.run.smithi042.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-16T12:55:57.856 INFO:teuthology.orchestra.run.smithi042.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-16T12:55:57.856 INFO:teuthology.orchestra.run.smithi042.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-16T12:55:57.857 INFO:teuthology.orchestra.run.smithi042.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-16T12:55:57.914 INFO:teuthology.orchestra.run.smithi042.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-16T12:55:57.914 INFO:teuthology.orchestra.run.smithi042.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-16T12:55:57.915 INFO:teuthology.orchestra.run.smithi042.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-16T12:55:57.916 INFO:teuthology.orchestra.run.smithi042.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-16T12:55:57.917 INFO:teuthology.orchestra.run.smithi042.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-16T12:55:57.918 INFO:teuthology.orchestra.run.smithi042.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-16T12:55:57.919 INFO:teuthology.orchestra.run.smithi042.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-16T12:55:57.922 INFO:teuthology.orchestra.run.smithi042.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-16T12:55:57.931 INFO:teuthology.orchestra.run.smithi042.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-16T12:55:57.942 INFO:teuthology.orchestra.run.smithi042.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-16T12:55:57.943 INFO:teuthology.orchestra.run.smithi042.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-16T12:55:57.948 INFO:teuthology.orchestra.run.smithi042.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-16T12:55:57.951 INFO:teuthology.orchestra.run.smithi042.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-16T12:55:57.960 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-16T12:55:57.968 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-16T12:55:57.968 INFO:teuthology.orchestra.run.smithi042.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-16T12:55:57.970 INFO:teuthology.orchestra.run.smithi042.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-16T12:55:58.007 INFO:teuthology.orchestra.run.smithi042.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-16T12:55:58.008 INFO:teuthology.orchestra.run.smithi042.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-16T12:55:58.008 INFO:teuthology.orchestra.run.smithi042.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-16T12:55:58.008 INFO:teuthology.orchestra.run.smithi042.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-16T12:55:58.009 INFO:teuthology.orchestra.run.smithi042.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-16T12:55:58.009 INFO:teuthology.orchestra.run.smithi042.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-16T12:55:58.010 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-16T12:55:58.021 INFO:teuthology.orchestra.run.smithi042.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T12:55:58.021 INFO:teuthology.orchestra.run.smithi042.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-16T12:55:58.021 INFO:teuthology.orchestra.run.smithi042.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-16T12:55:58.024 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-16T12:55:58.025 INFO:teuthology.orchestra.run.smithi042.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-16T12:55:58.042 INFO:teuthology.orchestra.run.smithi042.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-16T12:55:58.042 INFO:teuthology.orchestra.run.smithi042.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-16T12:55:58.043 INFO:teuthology.orchestra.run.smithi042.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-16T12:55:58.045 INFO:teuthology.orchestra.run.smithi042.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-16T12:55:58.045 INFO:teuthology.orchestra.run.smithi042.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-16T12:55:58.149 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-16T12:55:58.155 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-16T12:55:58.184 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-16T12:55:58.324 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-16T12:55:58.332 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-16T12:55:58.354 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-16T12:55:58.361 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-16T12:55:58.362 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-16T12:55:58.369 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-16T12:55:58.370 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-16T12:55:58.371 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-16T12:55:58.372 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-16T12:55:58.381 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-16T12:55:58.408 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-16T12:55:58.420 INFO:teuthology.orchestra.run.smithi042.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-16T12:55:58.462 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-16T12:55:58.463 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-16T12:55:58.464 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-16T12:55:58.464 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-16T12:55:58.466 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T12:55:58.467 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-16T12:55:58.468 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-16T12:55:58.470 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-16T12:55:58.487 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-16T12:55:58.552 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-16T12:55:58.555 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-16T12:55:58.556 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-16T12:55:58.564 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-16T12:55:58.565 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-16T12:55:58.566 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-16T12:55:58.566 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-16T12:55:58.569 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-16T12:55:58.572 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-16T12:55:58.628 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-16T12:55:58.650 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T12:55:58.650 INFO:teuthology.orchestra.run.smithi136.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-16T12:55:58.653 INFO:teuthology.orchestra.run.smithi136.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-16T12:55:58.655 INFO:teuthology.orchestra.run.smithi136.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-16T12:55:58.657 INFO:teuthology.orchestra.run.smithi136.stdout:Get:44 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-16T12:55:58.657 INFO:teuthology.orchestra.run.smithi136.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-16T12:55:58.658 INFO:teuthology.orchestra.run.smithi136.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-16T12:55:58.659 INFO:teuthology.orchestra.run.smithi136.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-16T12:55:58.667 INFO:teuthology.orchestra.run.smithi042.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-16T12:55:58.677 INFO:teuthology.orchestra.run.smithi136.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-16T12:55:58.678 INFO:teuthology.orchestra.run.smithi136.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-16T12:55:58.694 INFO:teuthology.orchestra.run.smithi136.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-16T12:55:58.694 INFO:teuthology.orchestra.run.smithi042.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-16T12:55:58.695 INFO:teuthology.orchestra.run.smithi136.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-16T12:55:58.714 INFO:teuthology.orchestra.run.smithi042.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-16T12:55:58.714 INFO:teuthology.orchestra.run.smithi136.stdout:Get:52 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-16T12:55:58.715 INFO:teuthology.orchestra.run.smithi042.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-16T12:55:58.731 INFO:teuthology.orchestra.run.smithi042.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-16T12:55:58.732 INFO:teuthology.orchestra.run.smithi042.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-16T12:55:58.751 INFO:teuthology.orchestra.run.smithi042.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-16T12:55:58.906 INFO:teuthology.orchestra.run.smithi136.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-16T12:55:58.908 INFO:teuthology.orchestra.run.smithi136.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-16T12:55:58.926 INFO:teuthology.orchestra.run.smithi136.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-16T12:55:58.942 INFO:teuthology.orchestra.run.smithi042.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-16T12:55:58.945 INFO:teuthology.orchestra.run.smithi042.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-16T12:55:58.965 INFO:teuthology.orchestra.run.smithi042.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-16T12:55:58.987 INFO:teuthology.orchestra.run.smithi136.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-16T12:55:59.008 INFO:teuthology.orchestra.run.smithi136.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-16T12:55:59.011 INFO:teuthology.orchestra.run.smithi136.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-16T12:55:59.085 INFO:teuthology.orchestra.run.smithi136.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-16T12:55:59.085 INFO:teuthology.orchestra.run.smithi136.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-16T12:55:59.085 INFO:teuthology.orchestra.run.smithi136.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-16T12:55:59.086 INFO:teuthology.orchestra.run.smithi136.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-16T12:55:59.086 INFO:teuthology.orchestra.run.smithi136.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-16T12:55:59.086 INFO:teuthology.orchestra.run.smithi136.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-16T12:55:59.087 INFO:teuthology.orchestra.run.smithi136.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-16T12:55:59.151 INFO:teuthology.orchestra.run.smithi136.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-16T12:55:59.151 INFO:teuthology.orchestra.run.smithi136.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-16T12:55:59.233 INFO:teuthology.orchestra.run.smithi136.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-16T12:55:59.237 INFO:teuthology.orchestra.run.smithi136.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-16T12:55:59.240 INFO:teuthology.orchestra.run.smithi136.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-16T12:55:59.243 INFO:teuthology.orchestra.run.smithi136.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-16T12:55:59.244 INFO:teuthology.orchestra.run.smithi136.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-16T12:55:59.248 INFO:teuthology.orchestra.run.smithi136.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-16T12:55:59.250 INFO:teuthology.orchestra.run.smithi136.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-16T12:55:59.323 INFO:teuthology.orchestra.run.smithi136.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-16T12:55:59.328 INFO:teuthology.orchestra.run.smithi136.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-16T12:55:59.489 INFO:teuthology.orchestra.run.smithi136.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-16T12:55:59.490 INFO:teuthology.orchestra.run.smithi136.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-16T12:55:59.490 INFO:teuthology.orchestra.run.smithi136.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-16T12:55:59.491 INFO:teuthology.orchestra.run.smithi136.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-16T12:55:59.491 INFO:teuthology.orchestra.run.smithi136.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-16T12:55:59.492 INFO:teuthology.orchestra.run.smithi136.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-16T12:55:59.504 INFO:teuthology.orchestra.run.smithi136.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T12:55:59.504 INFO:teuthology.orchestra.run.smithi136.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-16T12:55:59.505 INFO:teuthology.orchestra.run.smithi136.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-16T12:55:59.571 INFO:teuthology.orchestra.run.smithi136.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-16T12:55:59.652 INFO:teuthology.orchestra.run.smithi136.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-16T12:55:59.653 INFO:teuthology.orchestra.run.smithi136.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-16T12:55:59.653 INFO:teuthology.orchestra.run.smithi136.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-16T12:55:59.657 INFO:teuthology.orchestra.run.smithi136.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-16T12:55:59.658 INFO:teuthology.orchestra.run.smithi136.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-16T12:55:59.913 INFO:teuthology.orchestra.run.smithi136.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-16T12:55:59.946 INFO:teuthology.orchestra.run.smithi042.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-16T12:55:59.973 INFO:teuthology.orchestra.run.smithi136.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-16T12:55:59.977 INFO:teuthology.orchestra.run.smithi136.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-16T12:55:59.979 INFO:teuthology.orchestra.run.smithi136.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-16T12:56:00.011 INFO:teuthology.orchestra.run.smithi136.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-16T12:56:00.033 INFO:teuthology.orchestra.run.smithi042.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-16T12:56:00.037 INFO:teuthology.orchestra.run.smithi042.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-16T12:56:00.039 INFO:teuthology.orchestra.run.smithi042.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-16T12:56:00.105 INFO:teuthology.orchestra.run.smithi042.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-16T12:56:00.237 INFO:teuthology.orchestra.run.smithi136.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-16T12:56:00.259 INFO:teuthology.orchestra.run.smithi042.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-16T12:56:00.972 INFO:teuthology.orchestra.run.smithi136.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-16T12:56:00.973 INFO:teuthology.orchestra.run.smithi136.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-16T12:56:00.987 INFO:teuthology.orchestra.run.smithi136.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-16T12:56:01.009 INFO:teuthology.orchestra.run.smithi136.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-16T12:56:01.010 INFO:teuthology.orchestra.run.smithi136.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-16T12:56:01.012 INFO:teuthology.orchestra.run.smithi136.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-16T12:56:01.126 INFO:teuthology.orchestra.run.smithi042.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-16T12:56:01.127 INFO:teuthology.orchestra.run.smithi042.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-16T12:56:01.140 INFO:teuthology.orchestra.run.smithi042.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-16T12:56:01.174 INFO:teuthology.orchestra.run.smithi042.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-16T12:56:01.194 INFO:teuthology.orchestra.run.smithi042.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-16T12:56:01.196 INFO:teuthology.orchestra.run.smithi042.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-16T12:56:01.209 INFO:teuthology.orchestra.run.smithi136.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-16T12:56:01.415 INFO:teuthology.orchestra.run.smithi042.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-16T12:56:01.451 INFO:teuthology.orchestra.run.smithi136.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-16T12:56:01.452 INFO:teuthology.orchestra.run.smithi136.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-16T12:56:01.701 INFO:teuthology.orchestra.run.smithi042.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-16T12:56:01.701 INFO:teuthology.orchestra.run.smithi042.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-16T12:56:03.376 INFO:teuthology.orchestra.run.smithi136.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-16T12:56:03.381 INFO:teuthology.orchestra.run.smithi136.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-16T12:56:03.381 INFO:teuthology.orchestra.run.smithi136.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-16T12:56:03.685 INFO:teuthology.orchestra.run.smithi042.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-16T12:56:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-16T12:56:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-16T12:56:03.752 INFO:teuthology.orchestra.run.smithi136.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-16T12:56:03.977 INFO:teuthology.orchestra.run.smithi042.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-16T12:56:04.413 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 201 MB in 6s (31.6 MB/s) 2024-09-16T12:56:04.584 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-16T12:56:04.640 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-16T12:56:04.640 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 201 MB in 7s (30.6 MB/s) 2024-09-16T12:56:04.644 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-16T12:56:04.733 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T12:56:04.841 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-16T12:56:04.897 INFO:teuthology.orchestra.run.smithi042.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-16T12:56:04.902 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-16T12:56:04.981 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T12:56:05.104 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:05.237 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T12:56:05.360 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:05.486 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T12:56:05.925 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:06.058 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T12:56:06.201 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:06.357 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T12:56:06.754 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T12:56:06.770 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:06.837 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T12:56:07.103 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:07.144 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:07.165 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rados. 2024-09-16T12:56:07.184 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:07.248 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:07.505 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rados. 2024-09-16T12:56:07.523 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:07.563 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:07.576 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T12:56:07.593 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:07.634 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:07.904 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T12:56:07.920 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:07.962 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:08.074 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T12:56:08.091 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:08.132 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:08.315 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T12:56:08.331 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:08.373 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:08.418 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T12:56:08.440 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:08.476 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:08.693 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T12:56:08.712 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-16T12:56:08.751 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-16T12:56:08.846 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T12:56:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:08.904 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:09.021 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T12:56:09.040 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:09.079 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:09.241 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T12:56:09.256 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-16T12:56:09.299 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-16T12:56:09.466 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-16T12:56:09.482 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-16T12:56:09.524 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T12:56:09.560 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T12:56:09.578 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:09.618 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:09.877 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-16T12:56:09.893 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-16T12:56:09.927 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T12:56:09.996 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-16T12:56:10.016 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-16T12:56:10.054 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T12:56:10.280 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T12:56:10.297 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-16T12:56:10.338 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T12:56:10.407 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-16T12:56:10.426 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-16T12:56:10.457 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T12:56:10.733 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T12:56:10.749 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-16T12:56:10.791 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T12:56:10.843 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T12:56:10.859 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-16T12:56:10.927 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T12:56:11.128 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T12:56:11.148 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-16T12:56:11.178 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T12:56:11.305 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T12:56:11.322 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-16T12:56:11.362 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T12:56:11.648 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T12:56:11.666 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T12:56:11.716 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:56:11.724 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T12:56:11.740 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-16T12:56:11.782 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T12:56:12.259 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T12:56:12.275 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T12:56:12.277 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T12:56:12.293 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T12:56:12.340 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:56:12.380 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:56:12.646 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T12:56:12.662 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T12:56:12.704 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:56:12.897 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T12:56:12.915 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T12:56:12.955 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:56:13.342 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T12:56:13.358 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-16T12:56:13.359 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T12:56:13.376 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-16T12:56:13.416 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T12:56:13.528 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:56:13.769 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librgw2. 2024-09-16T12:56:13.788 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:13.827 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:13.987 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-16T12:56:14.006 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-16T12:56:14.045 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T12:56:14.406 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librgw2. 2024-09-16T12:56:14.426 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:14.464 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:14.672 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T12:56:14.687 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:14.730 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:15.108 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liboath0. 2024-09-16T12:56:15.125 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-16T12:56:15.166 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-16T12:56:15.283 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T12:56:15.300 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:15.342 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:15.503 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T12:56:15.523 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:15.561 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:15.705 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liboath0. 2024-09-16T12:56:15.723 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-16T12:56:15.763 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-16T12:56:16.048 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-common. 2024-09-16T12:56:16.065 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:16.106 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:16.107 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T12:56:16.124 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:16.166 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:16.653 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-common. 2024-09-16T12:56:16.668 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:16.735 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:18.995 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-base. 2024-09-16T12:56:19.015 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:19.063 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:19.283 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-base. 2024-09-16T12:56:19.299 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:19.351 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:19.983 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cryptography. 2024-09-16T12:56:20.003 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-16T12:56:20.041 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T12:56:20.213 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cryptography. 2024-09-16T12:56:20.231 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-16T12:56:20.262 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T12:56:20.447 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T12:56:20.463 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-16T12:56:20.505 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-16T12:56:20.683 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T12:56:20.699 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-16T12:56:20.741 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-16T12:56:20.841 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-packaging. 2024-09-16T12:56:20.857 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-16T12:56:20.900 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-16T12:56:21.052 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-packaging. 2024-09-16T12:56:21.070 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-16T12:56:21.110 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-16T12:56:21.161 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T12:56:21.177 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:21.244 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:21.330 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T12:56:21.347 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:21.413 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:21.641 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T12:56:21.660 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-16T12:56:21.699 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T12:56:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T12:56:21.843 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-16T12:56:21.884 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T12:56:22.002 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T12:56:22.020 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-16T12:56:22.068 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-16T12:56:22.204 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T12:56:22.222 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-16T12:56:22.262 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-16T12:56:22.414 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T12:56:22.432 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-16T12:56:22.489 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-16T12:56:22.632 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T12:56:22.649 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-16T12:56:22.690 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-16T12:56:22.784 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jwt. 2024-09-16T12:56:22.804 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-16T12:56:22.866 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T12:56:23.018 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jwt. 2024-09-16T12:56:23.038 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-16T12:56:23.076 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T12:56:23.170 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T12:56:23.186 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-16T12:56:23.228 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-16T12:56:23.430 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T12:56:23.448 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-16T12:56:23.513 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-16T12:56:23.582 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T12:56:23.598 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-16T12:56:23.640 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-16T12:56:23.816 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T12:56:23.832 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-16T12:56:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-16T12:56:23.976 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-mako. 2024-09-16T12:56:23.993 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-16T12:56:24.034 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T12:56:24.227 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-mako. 2024-09-16T12:56:24.244 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-16T12:56:24.285 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T12:56:24.396 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T12:56:24.413 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T12:56:24.454 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T12:56:24.647 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T12:56:24.664 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T12:56:24.705 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T12:56:24.757 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T12:56:24.773 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-16T12:56:24.815 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T12:56:25.038 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T12:56:25.056 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-16T12:56:25.096 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T12:56:25.144 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webob. 2024-09-16T12:56:25.161 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-16T12:56:25.202 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T12:56:25.441 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-webob. 2024-09-16T12:56:25.458 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-16T12:56:25.500 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T12:56:25.547 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-16T12:56:25.564 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-16T12:56:25.605 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T12:56:25.844 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-16T12:56:25.861 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-16T12:56:25.902 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T12:56:25.941 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-bs4. 2024-09-16T12:56:25.957 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-16T12:56:26.000 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-16T12:56:26.240 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-bs4. 2024-09-16T12:56:26.258 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-16T12:56:26.320 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-16T12:56:26.361 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T12:56:26.377 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-16T12:56:26.422 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T12:56:26.717 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T12:56:26.733 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-16T12:56:26.756 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T12:56:26.773 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-16T12:56:26.778 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T12:56:26.814 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-16T12:56:27.103 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T12:56:27.122 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-16T12:56:27.151 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-paste. 2024-09-16T12:56:27.153 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-16T12:56:27.166 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-16T12:56:27.209 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T12:56:27.481 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-paste. 2024-09-16T12:56:27.496 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T12:56:27.502 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-16T12:56:27.513 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-16T12:56:27.539 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T12:56:27.579 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T12:56:27.860 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T12:56:27.876 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-16T12:56:27.882 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T12:56:27.898 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-16T12:56:27.917 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T12:56:27.982 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-16T12:56:28.241 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T12:56:28.260 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-16T12:56:28.285 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T12:56:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-16T12:56:28.301 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-16T12:56:28.364 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-16T12:56:28.611 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T12:56:28.630 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-16T12:56:28.660 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-16T12:56:29.297 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T12:56:29.315 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-16T12:56:29.447 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T12:56:29.463 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-16T12:56:29.497 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T12:56:29.787 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T12:56:29.881 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T12:56:29.898 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-16T12:56:29.939 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T12:56:30.217 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T12:56:30.233 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-16T12:56:30.275 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T12:56:30.326 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T12:56:30.342 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:30.384 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:30.654 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T12:56:30.673 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:30.732 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:30.779 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T12:56:30.796 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:30.837 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:31.124 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T12:56:31.142 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:31.182 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:31.357 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T12:56:31.373 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:31.407 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:31.702 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T12:56:31.719 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:31.902 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:32.252 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T12:56:32.269 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:32.335 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:32.780 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T12:56:32.797 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:32.839 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:34.473 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph. 2024-09-16T12:56:34.489 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:34.531 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:34.909 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T12:56:34.925 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:34.967 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:35.059 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph. 2024-09-16T12:56:35.078 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:35.117 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:35.412 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T12:56:35.429 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:35.461 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T12:56:35.470 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:35.477 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:35.544 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:35.981 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T12:56:35.997 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:36.015 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package cephadm. 2024-09-16T12:56:36.031 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:36.031 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:36.073 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:36.351 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T12:56:36.367 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-16T12:56:36.409 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T12:56:36.500 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package cephadm. 2024-09-16T12:56:36.519 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:36.550 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:36.763 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T12:56:36.781 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:36.821 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:36.878 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T12:56:36.895 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-16T12:56:36.961 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T12:56:37.191 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T12:56:37.207 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T12:56:37.240 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T12:56:37.331 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T12:56:37.348 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:37.406 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:37.485 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjs-underscore. 2024-09-16T12:56:37.504 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-16T12:56:37.535 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T12:56:37.767 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T12:56:37.784 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T12:56:37.813 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-16T12:56:37.825 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T12:56:37.829 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-16T12:56:37.879 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T12:56:38.111 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libjs-underscore. 2024-09-16T12:56:38.128 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-16T12:56:38.170 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T12:56:38.199 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-routes. 2024-09-16T12:56:38.216 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-16T12:56:38.282 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-16T12:56:38.480 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-16T12:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-16T12:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T12:56:38.619 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T12:56:38.634 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:38.677 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:38.867 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-routes. 2024-09-16T12:56:38.887 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-16T12:56:38.925 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-16T12:56:39.262 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T12:56:39.278 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:39.320 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:40.142 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-decorator. 2024-09-16T12:56:40.158 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-16T12:56:40.200 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T12:56:40.512 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-16T12:56:40.531 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-16T12:56:40.570 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T12:56:40.692 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-decorator. 2024-09-16T12:56:40.711 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-16T12:56:40.750 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T12:56:40.906 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-scipy. 2024-09-16T12:56:40.924 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-16T12:56:40.964 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-16T12:56:41.086 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-16T12:56:41.104 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-16T12:56:41.144 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T12:56:41.481 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-scipy. 2024-09-16T12:56:41.497 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-16T12:56:41.539 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-16T12:56:42.257 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-16T12:56:42.274 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-16T12:56:42.315 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T12:56:42.852 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T12:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-16T12:56:42.868 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-16T12:56:42.883 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-16T12:56:42.922 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T12:56:42.929 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-16T12:56:43.314 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T12:56:43.330 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-16T12:56:43.397 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T12:56:43.468 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T12:56:43.487 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-16T12:56:43.526 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-16T12:56:43.913 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T12:56:43.932 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-16T12:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T12:56:44.131 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T12:56:44.151 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:44.189 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:44.754 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T12:56:44.772 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:44.812 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:45.334 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T12:56:45.350 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-16T12:56:45.392 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-16T12:56:45.712 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T12:56:45.731 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-16T12:56:45.770 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T12:56:46.115 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T12:56:46.131 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-16T12:56:46.173 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T12:56:46.174 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T12:56:46.195 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-16T12:56:46.232 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-16T12:56:46.501 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T12:56:46.517 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-16T12:56:46.559 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-16T12:56:46.572 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T12:56:46.588 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-16T12:56:46.627 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T12:56:46.906 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T12:56:46.922 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-16T12:56:46.963 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-16T12:56:47.014 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T12:56:47.033 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-16T12:56:47.097 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T12:56:47.309 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-blinker. 2024-09-16T12:56:47.325 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-16T12:56:47.366 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T12:56:47.458 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T12:56:47.478 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-16T12:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-16T12:56:47.694 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-16T12:56:47.713 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-16T12:56:47.744 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T12:56:47.853 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T12:56:47.869 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-16T12:56:47.912 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-16T12:56:48.082 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T12:56:48.101 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-16T12:56:48.165 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T12:56:48.257 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-blinker. 2024-09-16T12:56:48.276 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-16T12:56:48.315 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T12:56:48.443 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T12:56:48.459 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-16T12:56:48.501 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T12:56:48.642 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-16T12:56:48.663 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-16T12:56:48.700 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T12:56:48.847 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T12:56:48.865 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-16T12:56:48.905 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T12:56:49.071 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T12:56:49.088 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-16T12:56:49.130 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T12:56:49.452 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T12:56:49.471 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-16T12:56:49.527 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T12:56:49.549 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T12:56:49.566 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:49.607 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:49.821 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T12:56:49.837 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-16T12:56:49.901 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T12:56:50.135 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T12:56:50.156 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-16T12:56:50.193 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-16T12:56:50.554 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T12:56:50.555 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T12:56:50.571 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T12:56:50.572 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:50.613 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T12:56:50.631 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:50.891 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package jq. 2024-09-16T12:56:50.907 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T12:56:50.949 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T12:56:50.993 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T12:56:51.012 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-16T12:56:51.059 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-16T12:56:51.228 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package socat. 2024-09-16T12:56:51.246 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-16T12:56:51.286 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-16T12:56:51.429 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T12:56:51.447 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T12:56:51.487 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T12:56:51.626 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T12:56:51.644 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-16T12:56:51.683 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-16T12:56:51.777 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package jq. 2024-09-16T12:56:51.796 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T12:56:51.835 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T12:56:52.058 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-test. 2024-09-16T12:56:52.079 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:52.116 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:52.129 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package socat. 2024-09-16T12:56:52.148 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-16T12:56:52.187 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-16T12:56:52.510 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T12:56:52.526 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-16T12:56:52.568 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-16T12:56:52.932 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-test. 2024-09-16T12:56:52.951 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:52.990 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:56.653 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T12:56:56.672 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:56.710 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:57.051 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-16T12:56:57.069 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T12:56:57.109 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T12:56:57.621 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T12:56:57.640 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:57.679 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T12:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T12:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:58.091 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T12:56:58.106 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-16T12:56:58.165 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T12:56:58.174 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-16T12:56:58.192 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T12:56:58.232 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T12:56:58.519 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-16T12:56:58.536 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T12:56:58.569 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T12:56:58.752 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T12:56:58.771 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:56:58.810 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:56:59.263 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T12:56:59.281 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-16T12:56:59.339 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T12:56:59.685 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-16T12:56:59.702 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T12:56:59.743 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T12:57:00.277 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-16T12:57:00.297 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-16T12:57:00.334 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-16T12:57:00.687 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-16T12:57:00.708 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-16T12:57:00.745 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T12:57:01.082 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-attr. 2024-09-16T12:57:01.098 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-16T12:57:01.139 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-16T12:57:01.455 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-16T12:57:01.477 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-16T12:57:01.501 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webencodings. 2024-09-16T12:57:01.518 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-16T12:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-16T12:57:01.559 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T12:57:01.891 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-16T12:57:01.895 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-html5lib. 2024-09-16T12:57:01.909 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-16T12:57:01.911 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-16T12:57:01.949 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T12:57:01.953 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-16T12:57:02.277 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-attr. 2024-09-16T12:57:02.294 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-16T12:57:02.314 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-16T12:57:02.333 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-16T12:57:02.334 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-16T12:57:02.372 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T12:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-webencodings. 2024-09-16T12:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-16T12:57:02.746 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T12:57:02.876 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T12:57:02.894 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T12:57:02.934 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T12:57:03.082 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-html5lib. 2024-09-16T12:57:03.103 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-16T12:57:03.140 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-16T12:57:03.279 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T12:57:03.295 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-16T12:57:03.337 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-16T12:57:03.501 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-16T12:57:03.518 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-16T12:57:03.561 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T12:57:03.681 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T12:57:03.698 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-16T12:57:03.739 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T12:57:04.030 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T12:57:04.048 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T12:57:04.087 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T12:57:04.109 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-py. 2024-09-16T12:57:04.126 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-16T12:57:04.167 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T12:57:04.449 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T12:57:04.466 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-16T12:57:04.503 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T12:57:04.507 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-16T12:57:04.520 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-16T12:57:04.578 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T12:57:04.827 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T12:57:04.844 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-16T12:57:04.885 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T12:57:04.914 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T12:57:04.931 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-16T12:57:04.972 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T12:57:05.247 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-py. 2024-09-16T12:57:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-16T12:57:05.305 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T12:57:05.308 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T12:57:05.326 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-16T12:57:05.366 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-16T12:57:05.675 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T12:57:05.693 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-16T12:57:05.722 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T12:57:05.741 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-16T12:57:05.749 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T12:57:05.780 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T12:57:06.092 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T12:57:06.097 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T12:57:06.110 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-16T12:57:06.114 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-16T12:57:06.150 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T12:57:06.155 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T12:57:06.483 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T12:57:06.501 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-16T12:57:06.541 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-16T12:57:06.920 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T12:57:06.929 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package radosgw. 2024-09-16T12:57:06.938 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-16T12:57:06.946 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:57:06.977 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T12:57:06.986 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:07.281 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T12:57:07.298 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-16T12:57:07.339 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T12:57:08.134 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package radosgw. 2024-09-16T12:57:08.153 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:57:08.192 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:08.765 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T12:57:08.782 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:57:08.822 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:09.137 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-16T12:57:09.461 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T12:57:09.589 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T12:57:09.715 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T12:57:09.941 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:09.982 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T12:57:09.999 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T12:57:10.039 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:10.383 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-16T12:57:10.552 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user cephadm....done 2024-09-16T12:57:10.686 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T12:57:10.739 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T12:57:10.862 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T12:57:10.865 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T12:57:10.988 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T12:57:11.215 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:11.218 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T12:57:11.502 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T12:57:11.661 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-16T12:57:11.787 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T12:57:11.828 INFO:teuthology.orchestra.run.smithi042.stdout:Adding system user cephadm....done 2024-09-16T12:57:12.014 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T12:57:12.122 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-16T12:57:12.140 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T12:57:12.525 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T12:57:12.557 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T12:57:12.833 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-16T12:57:12.860 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T12:57:13.019 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-16T12:57:13.151 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T12:57:13.154 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T12:57:13.444 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T12:57:13.480 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-16T12:57:13.761 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:13.899 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T12:57:14.062 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T12:57:14.213 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-16T12:57:14.233 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-16T12:57:14.514 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T12:57:14.543 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T12:57:14.836 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T12:57:14.840 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T12:57:14.991 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T12:57:15.145 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:15.359 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-16T12:57:15.455 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T12:57:15.580 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-16T12:57:15.835 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T12:57:16.025 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T12:57:16.119 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T12:57:16.487 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T12:57:16.708 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T12:57:16.780 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T12:57:16.833 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T12:57:17.122 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-16T12:57:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-16T12:57:17.449 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-16T12:57:17.607 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T12:57:17.758 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T12:57:17.883 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T12:57:17.904 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T12:57:18.043 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T12:57:18.322 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T12:57:18.350 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T12:57:18.631 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T12:57:18.787 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T12:57:18.999 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-16T12:57:19.181 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T12:57:19.342 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-16T12:57:19.565 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-16T12:57:19.676 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T12:57:19.699 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:19.835 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T12:57:19.961 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T12:57:20.009 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:57:20.168 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-16T12:57:20.257 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T12:57:20.544 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T12:57:20.656 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T12:57:20.670 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-16T12:57:21.032 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T12:57:21.214 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-16T12:57:21.409 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-16T12:57:21.505 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T12:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:21.832 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:57:21.903 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:57:21.958 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T12:57:22.037 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-16T12:57:22.083 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T12:57:22.389 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T12:57:22.435 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T12:57:22.515 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-16T12:57:22.666 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-16T12:57:22.727 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-16T12:57:22.975 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T12:57:23.020 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T12:57:23.171 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T12:57:23.310 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:57:23.436 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T12:57:23.447 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-16T12:57:23.570 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T12:57:23.773 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T12:57:23.939 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T12:57:24.074 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T12:57:24.273 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-16T12:57:24.470 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-16T12:57:24.599 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T12:57:24.749 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-16T12:57:24.750 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T12:57:24.877 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T12:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-16T12:57:25.203 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-16T12:57:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T12:57:25.571 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T12:57:25.711 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T12:57:25.731 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T12:57:25.857 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T12:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-16T12:57:26.183 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T12:57:26.350 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-16T12:57:26.476 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T12:57:26.492 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:26.618 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-16T12:57:26.844 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-16T12:57:27.278 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T12:57:27.404 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T12:57:27.531 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T12:57:27.882 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T12:57:28.075 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:28.200 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:28.234 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T12:57:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-16T12:57:28.360 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-16T12:57:28.737 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-16T12:57:29.046 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-16T12:57:29.331 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:57:29.490 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-16T12:57:29.842 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:29.874 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:29.984 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T12:57:30.025 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T12:57:30.176 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-16T12:57:30.328 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:30.454 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T12:57:30.561 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-16T12:57:30.797 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T12:57:30.878 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-16T12:57:31.180 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T12:57:31.306 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-16T12:57:31.649 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:31.792 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T12:57:31.836 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:31.962 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T12:57:32.121 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-16T12:57:32.152 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:32.263 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T12:57:32.311 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T12:57:32.389 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-16T12:57:32.671 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T12:57:32.724 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T12:57:32.883 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-16T12:57:33.185 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:33.335 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:33.557 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T12:57:33.557 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T12:57:33.777 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:33.903 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T12:57:34.004 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:34.054 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-16T12:57:34.180 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T12:57:34.223 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T12:57:34.306 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-16T12:57:34.566 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T12:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T12:57:34.800 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-16T12:57:34.893 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T12:57:35.126 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:35.270 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:35.413 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:35.484 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T12:57:35.484 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T12:57:35.539 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:35.665 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:35.817 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-16T12:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:36.136 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-16T12:57:36.182 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T12:57:36.459 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T12:57:36.460 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:36.578 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:36.704 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T12:57:36.802 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T12:57:37.246 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:37.372 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:37.507 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:37.599 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:37.633 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:37.734 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-16T12:57:38.044 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-16T12:57:38.177 INFO:teuthology.orchestra.run.smithi136.stdout:Adding group ceph....done 2024-09-16T12:57:38.354 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:38.480 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:38.596 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user ceph....done 2024-09-16T12:57:38.606 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T12:57:38.739 INFO:teuthology.orchestra.run.smithi136.stdout:Setting system user ceph properties....done 2024-09-16T12:57:38.763 INFO:teuthology.orchestra.run.smithi136.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T12:57:38.912 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T12:57:39.231 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T12:57:39.585 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:39.677 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:39.802 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:40.076 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T12:57:40.076 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T12:57:40.162 INFO:teuthology.orchestra.run.smithi042.stdout:Adding group ceph....done 2024-09-16T12:57:40.531 INFO:teuthology.orchestra.run.smithi042.stdout:Adding system user ceph....done 2024-09-16T12:57:40.708 INFO:teuthology.orchestra.run.smithi042.stdout:Setting system user ceph properties....done 2024-09-16T12:57:40.728 INFO:teuthology.orchestra.run.smithi042.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T12:57:40.729 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:40.883 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T12:57:41.141 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T12:57:41.204 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T12:57:41.607 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:41.687 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:41.833 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:41.878 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T12:57:41.879 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T12:57:42.134 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T12:57:42.134 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T12:57:42.308 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:42.541 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T12:57:42.541 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T12:57:42.790 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:43.001 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:43.163 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T12:57:43.487 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T12:57:43.487 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T12:57:43.643 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T12:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T12:57:43.936 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:44.062 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:44.205 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:44.344 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:44.459 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T12:57:44.459 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T12:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T12:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T12:57:44.914 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:45.045 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:45.057 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:45.216 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:45.359 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:45.486 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-16T12:57:45.555 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T12:57:45.555 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T12:57:45.662 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-16T12:57:45.947 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-16T12:57:46.006 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:46.157 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:46.300 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:46.529 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T12:57:46.529 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T12:57:46.968 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:47.997 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T12:57:48.133 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-16T12:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-16T12:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-16T12:57:48.892 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-16T12:57:49.526 INFO:teuthology.orchestra.run.smithi136.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T12:57:49.530 DEBUG:teuthology.parallel:result is None 2024-09-16T12:57:51.472 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-16T12:57:52.128 INFO:teuthology.orchestra.run.smithi042.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T12:57:52.132 DEBUG:teuthology.parallel:result is None 2024-09-16T12:57:52.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T12:57:52.276 DEBUG:teuthology.orchestra.run.smithi042:> dpkg-query -W -f '${Version}' ceph 2024-09-16T12:57:52.299 INFO:teuthology.orchestra.run.smithi042.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-16T12:57:52.300 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-16T12:57:52.300 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-16T12:57:52.302 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T12:57:52.449 DEBUG:teuthology.orchestra.run.smithi136:> dpkg-query -W -f '${Version}' ceph 2024-09-16T12:57:52.471 INFO:teuthology.orchestra.run.smithi136.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-16T12:57:52.472 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-16T12:57:52.472 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-16T12:57:52.473 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T12:57:52.474 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:57:52.474 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T12:57:52.491 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:57:52.491 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T12:57:52.527 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T12:57:52.527 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:57:52.527 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T12:57:52.543 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T12:57:52.559 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:57:52.559 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T12:57:52.575 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T12:57:52.633 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T12:57:52.634 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:57:52.634 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T12:57:52.650 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T12:57:52.709 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:57:52.709 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T12:57:52.725 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T12:57:52.740 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T12:57:52.741 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:57:52.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T12:57:52.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T12:57:52.766 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:57:52.766 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T12:57:52.798 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T12:57:52.807 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T12:57:52.878 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T12:57:52.879 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /var/log/ceph 2024-09-16T12:57:52.881 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /var/log/ceph 2024-09-16T12:57:52.896 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T12:57:52.896 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T12:57:52.939 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T12:57:52.954 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T12:57:52.954 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T12:57:52.956 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T12:57:52.975 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T12:57:52.976 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T12:57:52.976 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280', 'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'sangadi@redhat.com', 'first_in_suite': False, 'job_id': '7907280', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'owner': 'scheduled_sangadi@teuthology', 'priority': 130, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 9764, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'targets': {'smithi042.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi136.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'exec': {'client.0': ['sudo ceph osd pool create cache 4', 'sudo ceph osd tier add rbd cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay rbd cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 60', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_10:44:09', 'tube': 'smithi', 'user': 'sangadi', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T12:57:52.976 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T12:57:52.978 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T12:57:52.983 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T12:57:53.027 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T12:57:53.042 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:57:53.042 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T12:57:53.049 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T12:57:53.049 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-09-16T12:57:53.057 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T12:57:53.057 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T12:57:53.057 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-09-16T12:57:53.057 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T12:57:53.057 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-16 12:54:18.478316368 +0000 2024-09-16T12:57:53.057 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-16 12:54:18.474316461 +0000 2024-09-16T12:57:53.057 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-16 12:54:18.474316461 +0000 2024-09-16T12:57:53.057 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-09-16T12:57:53.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T12:57:53.112 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-16T12:57:53.112 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-16T12:57:53.112 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000308756 s, 1.7 MB/s 2024-09-16T12:57:53.113 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T12:57:53.121 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-09-16T12:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T12:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T12:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-09-16T12:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T12:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-16 12:54:19.162300594 +0000 2024-09-16T12:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-16 12:54:18.934305852 +0000 2024-09-16T12:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-16 12:54:18.934305852 +0000 2024-09-16T12:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-09-16T12:57:53.128 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T12:57:53.184 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-16T12:57:53.184 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-16T12:57:53.184 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.0002923 s, 1.8 MB/s 2024-09-16T12:57:53.186 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T12:57:53.194 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-09-16T12:57:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T12:57:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T12:57:53.200 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-09-16T12:57:53.200 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T12:57:53.200 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-16 12:54:19.598290541 +0000 2024-09-16T12:57:53.200 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-16 12:54:19.390295336 +0000 2024-09-16T12:57:53.200 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-16 12:54:19.390295336 +0000 2024-09-16T12:57:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-09-16T12:57:53.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T12:57:53.254 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-16T12:57:53.254 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-16T12:57:53.254 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000271814 s, 1.9 MB/s 2024-09-16T12:57:53.255 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T12:57:53.263 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-09-16T12:57:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T12:57:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T12:57:53.271 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 380 Links: 1 2024-09-16T12:57:53.271 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T12:57:53.271 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-16 12:54:20.042280301 +0000 2024-09-16T12:57:53.271 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-16 12:54:19.806285744 +0000 2024-09-16T12:57:53.271 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-16 12:54:19.806285744 +0000 2024-09-16T12:57:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-09-16T12:57:53.271 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T12:57:53.332 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-16T12:57:53.332 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-16T12:57:53.332 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000297083 s, 1.7 MB/s 2024-09-16T12:57:53.333 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T12:57:53.341 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T12:57:53.341 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:57:53.341 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T12:57:53.348 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T12:57:53.348 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-09-16T12:57:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T12:57:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T12:57:53.397 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 370 Links: 1 2024-09-16T12:57:53.397 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T12:57:53.397 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-16 12:54:17.993378455 +0000 2024-09-16T12:57:53.397 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-16 12:54:17.769383618 +0000 2024-09-16T12:57:53.397 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-16 12:54:17.769383618 +0000 2024-09-16T12:57:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-16T12:57:53.397 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T12:57:53.408 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-16T12:57:53.408 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-16T12:57:53.408 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000440384 s, 1.2 MB/s 2024-09-16T12:57:53.409 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T12:57:53.416 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-09-16T12:57:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T12:57:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T12:57:53.465 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-09-16T12:57:53.465 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T12:57:53.466 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-16 12:54:18.425368498 +0000 2024-09-16T12:57:53.466 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-16 12:54:18.213373384 +0000 2024-09-16T12:57:53.466 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-16 12:54:18.213373384 +0000 2024-09-16T12:57:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-16T12:57:53.466 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T12:57:53.478 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-16T12:57:53.478 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-16T12:57:53.478 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000412644 s, 1.2 MB/s 2024-09-16T12:57:53.479 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T12:57:53.488 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-09-16T12:57:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T12:57:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T12:57:53.537 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 383 Links: 1 2024-09-16T12:57:53.537 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T12:57:53.537 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-16 12:54:18.677362689 +0000 2024-09-16T12:57:53.537 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-16 12:54:18.673362781 +0000 2024-09-16T12:57:53.537 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-16 12:54:18.673362781 +0000 2024-09-16T12:57:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-16T12:57:53.538 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T12:57:53.549 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-16T12:57:53.549 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-16T12:57:53.549 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000396841 s, 1.3 MB/s 2024-09-16T12:57:53.550 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T12:57:53.558 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-09-16T12:57:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T12:57:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T12:57:53.609 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 387 Links: 1 2024-09-16T12:57:53.609 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T12:57:53.609 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-16 12:54:19.285348675 +0000 2024-09-16T12:57:53.609 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-16 12:54:19.073353561 +0000 2024-09-16T12:57:53.609 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-16 12:54:19.073353561 +0000 2024-09-16T12:57:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-16T12:57:53.610 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T12:57:53.620 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-16T12:57:53.621 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-16T12:57:53.621 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000398538 s, 1.3 MB/s 2024-09-16T12:57:53.621 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T12:57:53.629 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T12:57:53.629 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T12:57:53.629 INFO:tasks.ceph:Generating config... 2024-09-16T12:57:53.630 INFO:tasks.ceph:[client] rbd cache = True 2024-09-16T12:57:53.630 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-09-16T12:57:53.630 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-16T12:57:53.630 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-16T12:57:53.630 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-16T12:57:53.631 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T12:57:53.631 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T12:57:53.631 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T12:57:53.631 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T12:57:53.631 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T12:57:53.632 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T12:57:53.633 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-16T12:57:53.633 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T12:57:53.633 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-16T12:57:53.633 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T12:57:53.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T12:57:53.667 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T12:57:53.670 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T12:57:53.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T12:57:53.760 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.42'), ('mon.c', '[v2:172.21.15.42:3301,v1:172.21.15.42:6790]'), ('mon.b', '172.21.15.136')] 2024-09-16T12:57:53.760 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.136', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-16T12:57:53.762 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:57:53.762 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T12:57:53.768 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.42 --addv c '[v2:172.21.15.42:3301,v1:172.21.15.42:6790]' --add b 172.21.15.136 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:57:53.850 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:57:53.850 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: generated fsid c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7 2024-09-16T12:57:53.850 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = octopus 2024-09-16T12:57:53.850 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 0 2024-09-16T12:57:53.850 INFO:teuthology.orchestra.run.smithi042.stdout:fsid c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7 2024-09-16T12:57:53.850 INFO:teuthology.orchestra.run.smithi042.stdout:last_changed 2024-09-16T12:57:53.847361+0000 2024-09-16T12:57:53.850 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T12:57:53.847361+0000 2024-09-16T12:57:53.851 INFO:teuthology.orchestra.run.smithi042.stdout:min_mon_release 15 (octopus) 2024-09-16T12:57:53.851 INFO:teuthology.orchestra.run.smithi042.stdout:election_strategy: 1 2024-09-16T12:57:53.851 INFO:teuthology.orchestra.run.smithi042.stdout:0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-09-16T12:57:53.851 INFO:teuthology.orchestra.run.smithi042.stdout:1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-09-16T12:57:53.851 INFO:teuthology.orchestra.run.smithi042.stdout:2: [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] mon.c 2024-09-16T12:57:53.851 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T12:57:53.852 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T12:57:53.857 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7... 2024-09-16T12:57:53.858 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T12:57:53.860 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T12:57:53.893 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: chdir = "" 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: auth supported = cephx 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: filestore xattr use omap = true 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: mon clock drift allowed = 1.000 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: osd crush chooseleaf type = 0 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: auth debug = true 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on old message = true 2024-09-16T12:57:53.894 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on bug = true 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: mon pg warn max object skew = 0 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default size = 2 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow primary affinity = true 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow pg remap = true 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on legacy crush tunables = false 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on crush straw calc version zero = false 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on no sortbitwise = false 2024-09-16T12:57:53.895 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on osd down out interval zero = false 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on too few osds = false 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: mon_allow_pool_size_one = true 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: osd default data pool replay window = 5 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: mon allow pool delete = true 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: mon cluster log file level = debug 2024-09-16T12:57:53.896 INFO:teuthology.orchestra.run.smithi042.stdout: debug asserts on shutdown = true 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: mon health detail to clog = false 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: mon host = "172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.136" 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: mon client directed command retry = 5 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on pool no app = False 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject socket failures = 5000 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout:[osd] 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: osd journal size = 100 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub load threshold = 5.0 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub max interval = 600 2024-09-16T12:57:53.897 INFO:teuthology.orchestra.run.smithi042.stdout: osd mclock profile = high_recovery_ops 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: osd recover clone overlap = true 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: osd recovery max chunk = 1048576 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug shutdown = true 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug op order = true 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify stray on activate = true 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: osd open classes on start = true 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug pg log writeout = true 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: osd deep scrub update digest min age = 30 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.898 INFO:teuthology.orchestra.run.smithi042.stdout: osd map max advance = 10 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: journal zero on create = true 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: filestore ondisk finisher threads = 3 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: filestore apply finisher threads = 3 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: bdev debug aio = true 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug misdirected ops = true 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore block size = 96636764160 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore compression algorithm = lz4 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore compression mode = aggressive 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore fsck on mount = True 2024-09-16T12:57:53.899 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluefs = 1/20 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluestore = 1/20 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: debug osd = 20 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: debug rocksdb = 4/10 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd full ratio = 0.9 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd nearfull ratio = 0.8 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: osd failsafe full ratio = 0.95 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: osd max markdown count = 1000 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: osd objectstore = bluestore 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: osd shutdown pgref assert = True 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr] 2024-09-16T12:57:53.900 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: debug mgr = 20 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: mgr/telemetry/nag = false 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout:[mon] 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: debug paxos = 20 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-09-16T12:57:53.901 INFO:teuthology.orchestra.run.smithi042.stdout: mon data avail warn = 5 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: mon mgr mkfs grace = 240 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd reporter subtree level = osd 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd prime pg temp = true 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T12:57:53.902 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: # 1m isn't quite enough 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: mon_down_mkfs_grace = 2m 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_filestore_osds = false 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout:[client] 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: rgw cache enabled = true 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable ops log = true 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable usage log = true 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T12:57:53.903 INFO:teuthology.orchestra.run.smithi042.stdout: rbd cache = True 2024-09-16T12:57:53.904 INFO:teuthology.orchestra.run.smithi042.stdout: rbd cache policy = writearound 2024-09-16T12:57:53.904 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.a] 2024-09-16T12:57:53.904 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.c] 2024-09-16T12:57:53.904 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.b] 2024-09-16T12:57:53.904 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2024-09-16T12:57:53.904 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2024-09-16T12:57:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: mon_allow_pool_size_one = true 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2024-09-16T12:57:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.136" 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: mon client directed command retry = 5 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on pool no app = False 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject socket failures = 5000 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 600 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: osd mclock profile = high_recovery_ops 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore compression algorithm = lz4 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore compression mode = aggressive 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 1/20 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 1/20 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 20 2024-09-16T12:57:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 4/10 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: osd max markdown count = 1000 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: osd shutdown pgref assert = True 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-09-16T12:57:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-09-16T12:57:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T12:57:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: # 1m isn't quite enough 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: mon_down_mkfs_grace = 2m 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_filestore_osds = false 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: rbd cache = True 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: rbd cache policy = writearound 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2024-09-16T12:57:53.914 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2024-09-16T12:57:53.915 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2024-09-16T12:57:53.916 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T12:57:53.916 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T12:57:54.005 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T12:57:54.006 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:57:54.006 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T12:57:54.012 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:57:54.012 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T12:57:54.019 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T12:57:54.019 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:57:54.019 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T12:57:54.019 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T12:57:54.088 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:57:54.089 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:57:54.095 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T12:57:54.095 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:57:54.096 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T12:57:54.096 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T12:57:54.122 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:57:54.122 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:57:54.127 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T12:57:54.127 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T12:57:54.127 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T12:57:54.172 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T12:57:54.175 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T12:57:54.221 INFO:teuthology.orchestra.run.smithi136.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T12:57:54.223 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T12:57:54.223 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T12:57:54.223 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T12:57:54.259 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T12:57:54.273 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T12:57:54.308 INFO:teuthology.orchestra.run.smithi136.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-16T12:57:54.321 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T12:57:54.322 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T12:57:54.322 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T12:57:54.338 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T12:57:54.338 INFO:tasks.ceph:role: osd.0 2024-09-16T12:57:54.339 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T12:57:54.339 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T12:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T12:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T12:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T12:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 2024-09-16T12:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T12:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-16T12:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T12:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T12:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T12:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T12:57:54.385 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-16T12:57:54.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T12:57:54.406 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T12:57:54.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T12:57:54.462 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T12:57:54.462 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T12:57:54.472 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T12:57:54.472 INFO:tasks.ceph:role: osd.1 2024-09-16T12:57:54.472 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T12:57:54.473 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T12:57:54.514 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T12:57:54.514 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T12:57:54.514 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T12:57:54.514 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 2024-09-16T12:57:54.514 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T12:57:54.514 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-16T12:57:54.514 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T12:57:54.515 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T12:57:54.515 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T12:57:54.515 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T12:57:54.516 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-16T12:57:54.516 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T12:57:54.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T12:57:54.541 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T12:57:54.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T12:57:54.542 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T12:57:54.602 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T12:57:54.602 INFO:tasks.ceph:role: osd.2 2024-09-16T12:57:54.602 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T12:57:54.602 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T12:57:54.645 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T12:57:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T12:57:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T12:57:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 2024-09-16T12:57:54.646 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T12:57:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-16T12:57:54.646 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T12:57:54.646 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T12:57:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T12:57:54.646 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T12:57:54.648 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-16T12:57:54.648 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T12:57:54.666 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T12:57:54.674 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T12:57:54.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T12:57:54.675 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T12:57:54.686 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T12:57:54.686 INFO:tasks.ceph:role: osd.3 2024-09-16T12:57:54.686 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T12:57:54.686 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T12:57:54.728 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T12:57:54.728 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T12:57:54.728 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T12:57:54.728 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 2024-09-16T12:57:54.728 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T12:57:54.728 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-16T12:57:54.728 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T12:57:54.729 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T12:57:54.729 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T12:57:54.729 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T12:57:54.730 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-16T12:57:54.730 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-16T12:57:54.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T12:57:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T12:57:54.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T12:57:54.806 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:57:54.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:54.861+0000 7fa0f4a4c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T12:57:54.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:54.861+0000 7fa0f4a4c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T12:57:54.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:54.861+0000 7fa0f4a4c540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T12:57:54.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:54.861+0000 7fa0f4a4c540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T12:57:54.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:54.861+0000 7fa0f4a4c540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T12:57:54.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:54.865+0000 7fa0f4a4c540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T12:57:55.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:55.424+0000 7fa0f4a4c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:57:56.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:56.204+0000 7fa0f4a4c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:57:56.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:56.988+0000 7fa0f4a4c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:57:57.850 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T12:57:57.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:57:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:57.896+0000 7fd308aaa540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T12:57:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:57.896+0000 7fd308aaa540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T12:57:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:57.896+0000 7fd308aaa540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T12:57:57.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:57.896+0000 7fd308aaa540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T12:57:57.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:57.896+0000 7fd308aaa540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T12:57:57.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:57.896+0000 7fd308aaa540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T12:57:58.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:58.448+0000 7fd308aaa540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:57:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:57:59.228+0000 7fd308aaa540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:00.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:00.012+0000 7fd308aaa540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:00.878 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T12:58:00.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:58:00.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:00.968+0000 7ff8e8b58540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T12:58:00.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:00.968+0000 7ff8e8b58540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T12:58:00.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:00.968+0000 7ff8e8b58540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T12:58:00.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:00.968+0000 7ff8e8b58540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T12:58:00.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:00.968+0000 7ff8e8b58540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T12:58:00.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:00.968+0000 7ff8e8b58540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T12:58:01.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:01.520+0000 7ff8e8b58540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:02.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:02.300+0000 7ff8e8b58540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:03.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:03.080+0000 7ff8e8b58540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:03.925 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T12:58:03.942 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:58:03.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:03.976+0000 7fc372d6f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T12:58:03.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:03.976+0000 7fc372d6f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T12:58:03.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:03.976+0000 7fc372d6f540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T12:58:03.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:03.976+0000 7fc372d6f540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T12:58:03.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:03.976+0000 7fc372d6f540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T12:58:03.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:03.976+0000 7fc372d6f540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T12:58:04.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:04.528+0000 7fc372d6f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:05.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:05.316+0000 7fc372d6f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:06.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T12:58:06.100+0000 7fc372d6f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:06.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T12:58:07.010 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T12:58:07.028 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T12:58:07.028 INFO:tasks.ceph:role: osd.4 2024-09-16T12:58:07.029 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T12:58:07.029 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T12:58:07.671 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T12:58:07.672 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T12:58:07.672 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T12:58:07.672 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2024-09-16T12:58:07.672 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T12:58:07.672 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-16T12:58:07.672 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T12:58:07.672 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T12:58:07.672 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T12:58:07.672 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T12:58:07.673 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-16T12:58:07.673 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-16T12:58:07.731 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T12:58:07.743 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T12:58:07.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T12:58:07.744 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T12:58:07.753 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T12:58:07.754 INFO:tasks.ceph:role: osd.5 2024-09-16T12:58:07.754 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T12:58:07.754 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T12:58:08.244 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T12:58:08.245 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T12:58:08.245 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T12:58:08.245 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2024-09-16T12:58:08.245 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T12:58:08.245 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-16T12:58:08.245 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T12:58:08.245 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T12:58:08.245 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T12:58:08.245 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T12:58:08.246 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-16T12:58:08.246 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T12:58:08.276 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T12:58:08.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T12:58:08.283 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T12:58:08.283 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T12:58:08.295 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T12:58:08.295 INFO:tasks.ceph:role: osd.6 2024-09-16T12:58:08.295 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T12:58:08.295 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T12:58:08.944 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T12:58:08.944 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T12:58:08.944 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T12:58:08.944 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2024-09-16T12:58:08.944 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T12:58:08.944 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-16T12:58:08.945 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T12:58:08.945 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T12:58:08.945 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T12:58:08.945 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T12:58:08.946 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-16T12:58:08.946 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-16T12:58:09.013 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T12:58:09.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T12:58:09.024 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T12:58:09.024 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T12:58:09.080 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T12:58:09.080 INFO:tasks.ceph:role: osd.7 2024-09-16T12:58:09.080 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T12:58:09.080 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T12:58:09.698 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T12:58:09.698 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T12:58:09.698 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T12:58:09.698 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2024-09-16T12:58:09.698 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T12:58:09.698 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-16T12:58:09.698 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T12:58:09.698 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T12:58:09.699 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T12:58:09.699 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T12:58:09.700 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-16T12:58:09.700 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-16T12:58:09.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T12:58:09.774 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T12:58:09.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T12:58:09.775 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:58:09.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:09.849+0000 7eff030cf540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T12:58:09.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:09.853+0000 7eff030cf540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T12:58:09.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:09.853+0000 7eff030cf540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T12:58:09.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:09.853+0000 7eff030cf540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T12:58:09.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:09.853+0000 7eff030cf540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T12:58:09.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:09.853+0000 7eff030cf540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T12:58:10.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:10.413+0000 7eff030cf540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:11.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:11.205+0000 7eff030cf540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:12.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:12.001+0000 7eff030cf540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:12.870 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T12:58:12.885 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:58:12.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:12.965+0000 7f403ecae540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T12:58:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:12.965+0000 7f403ecae540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T12:58:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:12.965+0000 7f403ecae540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T12:58:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:12.965+0000 7f403ecae540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T12:58:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:12.965+0000 7f403ecae540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T12:58:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:12.965+0000 7f403ecae540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T12:58:13.529 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:13.525+0000 7f403ecae540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:14.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:14.317+0000 7f403ecae540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:15.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:15.097+0000 7f403ecae540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:15.950 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T12:58:15.962 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:58:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:15.997+0000 7f4f9bfb9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T12:58:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:15.997+0000 7f4f9bfb9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T12:58:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:15.997+0000 7f4f9bfb9540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T12:58:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:15.997+0000 7f4f9bfb9540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T12:58:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:15.997+0000 7f4f9bfb9540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T12:58:16.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:15.997+0000 7f4f9bfb9540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T12:58:16.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:16.553+0000 7f4f9bfb9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:17.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:17.345+0000 7f4f9bfb9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:18.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:18.125+0000 7f4f9bfb9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:18.981 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T12:58:19.039 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:58:19.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:19.072+0000 7f75ae2c6540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T12:58:19.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:19.072+0000 7f75ae2c6540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T12:58:19.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:19.072+0000 7f75ae2c6540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T12:58:19.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:19.072+0000 7f75ae2c6540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T12:58:19.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:19.072+0000 7f75ae2c6540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T12:58:19.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:19.072+0000 7f75ae2c6540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T12:58:19.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:19.624+0000 7f75ae2c6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:20.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:20.408+0000 7f75ae2c6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:21.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T12:58:21.192+0000 7f75ae2c6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T12:58:22.062 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T12:58:22.079 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T12:58:22.079 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:58:22.079 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T12:58:22.096 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:58:22.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T12:58:22.151 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:58:22.151 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T12:58:22.166 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:58:22.166 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T12:58:22.181 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:58:22.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T12:58:22.238 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:58:22.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T12:58:22.254 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:58:22.254 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T12:58:22.264 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:58:22.265 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T12:58:22.275 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:58:22.276 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T12:58:22.291 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:58:22.291 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T12:58:22.347 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:58:22.347 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T12:58:22.354 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:58:22.354 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-16T12:58:22.360 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T12:58:22.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T12:58:22.400 DEBUG:teuthology.orchestra.run.smithi136:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T12:58:22.412 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.y] 2024-09-16T12:58:22.412 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBSK+hmWZUnChAA/UJEn9O362THM1eAJnyQHQ== 2024-09-16T12:58:22.412 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.0] 2024-09-16T12:58:22.412 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBSK+hmGTOlMxAAJYOQl7ircZuM7L6um0Iscw== 2024-09-16T12:58:22.412 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.1] 2024-09-16T12:58:22.412 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBVK+hmyxygNRAARplg51h/IMpFZRHpDj33AQ== 2024-09-16T12:58:22.412 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.2] 2024-09-16T12:58:22.412 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBYK+hmZNzwORAAl56lLWUNrVkAIALZtzSsrA== 2024-09-16T12:58:22.412 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.3] 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBbK+hmYrF0OhAAeJf0Og7Ef8blsSJuiazRCQ== 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.x] 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBSK+hmyY4fDRAAiX7aaNbk7+zyM6Aeqiv9rg== 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.4] 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBhK+hm7gbKMhAAxz0TjwgAYRO7JysaJ4X0LQ== 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.5] 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBkK+hmQtWNORAA8dtCBlWpBQiCwG3cUiITow== 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.6] 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBnK+hmCNN8OxAAE90rZGk842LPf0yaVsPW3g== 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.7] 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBrK+hmokh0BBAAMnldEjVXZ/vXEKKP1aga/w== 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBSK+hm3MxTDxAAWC5OSovGE00qEDQ5L/B3tg== 2024-09-16T12:58:22.413 INFO:teuthology.orchestra.run.smithi042.stdout:[client.1] 2024-09-16T12:58:22.414 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBSK+hmXsRSEhAAYrrsPzqfSJt0YUIzD479Yw== 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.y] 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBSK+hmWZUnChAA/UJEn9O362THM1eAJnyQHQ== 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.0] 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBSK+hmGTOlMxAAJYOQl7ircZuM7L6um0Iscw== 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.1] 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBVK+hmyxygNRAARplg51h/IMpFZRHpDj33AQ== 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.2] 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBYK+hmZNzwORAAl56lLWUNrVkAIALZtzSsrA== 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.3] 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBbK+hmYrF0OhAAeJf0Og7Ef8blsSJuiazRCQ== 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.x] 2024-09-16T12:58:22.415 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBSK+hmyY4fDRAAiX7aaNbk7+zyM6Aeqiv9rg== 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.4] 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBhK+hm7gbKMhAAxz0TjwgAYRO7JysaJ4X0LQ== 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.5] 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBkK+hmQtWNORAA8dtCBlWpBQiCwG3cUiITow== 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.6] 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBnK+hmCNN8OxAAE90rZGk842LPf0yaVsPW3g== 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.7] 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBrK+hmokh0BBAAMnldEjVXZ/vXEKKP1aga/w== 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout:[client.0] 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBSK+hm3MxTDxAAWC5OSovGE00qEDQ5L/B3tg== 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout:[client.1] 2024-09-16T12:58:22.416 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBSK+hmXsRSEhAAYrrsPzqfSJt0YUIzD479Yw== 2024-09-16T12:58:22.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T12:58:22.419 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T12:58:22.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.507 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.550 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.552 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.596 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.598 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.715 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.717 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.783 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T12:58:22.785 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T12:58:22.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.844 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.898 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.927 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:22.999 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:23.001 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:23.048 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:23.050 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T12:58:23.091 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T12:58:23.131 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T12:58:23.180 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T12:58:23.182 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T12:58:23.229 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T12:58:23.229 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T12:58:23.240 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T12:58:23.292 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T12:58:23.305 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T12:58:23.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T12:58:23.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T12:58:23.431 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T12:58:23.446 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T12:58:23.503 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T12:58:23.515 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:58:23.517 DEBUG:teuthology.orchestra.run.smithi136:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T12:58:23.522 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T12:58:23.523 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T12:58:23.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T12:58:23.568 INFO:tasks.ceph.mon.a:Started 2024-09-16T12:58:23.568 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T12:58:23.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T12:58:23.570 INFO:tasks.ceph.mon.c:Started 2024-09-16T12:58:23.570 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T12:58:23.570 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T12:58:23.573 INFO:tasks.ceph.mon.b:Started 2024-09-16T12:58:23.573 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T12:58:23.573 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T12:58:23.573 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T12:58:23.575 INFO:tasks.ceph.mgr.y:Started 2024-09-16T12:58:23.576 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T12:58:23.576 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T12:58:23.578 INFO:tasks.ceph.mgr.x:Started 2024-09-16T12:58:23.578 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T12:58:23.578 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T12:58:34.082 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-09-16T12:58:34.093 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T12:58:34.093 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:58:34.093 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T12:58:34.106 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:58:34.107 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T12:58:34.166 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:58:34.166 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T12:58:34.175 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T12:58:34.175 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T12:58:34.187 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:34.180+0000 7f0e49df3000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T12:58:34.189 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:58:34.189 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T12:58:34.191 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:34.188+0000 7f6de9003000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T12:58:34.205 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:58:34.205 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T12:58:34.259 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:58:34.259 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T12:58:34.273 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-16T12:58:34.274 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T12:58:34.287 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 0d8a8277-07cd-40f1-8c98-98b8b51d1689 0 2024-09-16T12:58:34.483 INFO:teuthology.orchestra.run.smithi136.stdout:0 2024-09-16T12:58:34.496 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 6ce5996a-5e0e-4c9d-b268-27b34a14d830 1 2024-09-16T12:58:34.696 INFO:teuthology.orchestra.run.smithi136.stdout:1 2024-09-16T12:58:34.709 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 156042ab-909e-4368-b510-84018cb4354f 2 2024-09-16T12:58:34.885 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:34.880+0000 7f6de9003000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T12:58:34.903 INFO:teuthology.orchestra.run.smithi136.stdout:2 2024-09-16T12:58:34.915 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new b6c53823-446f-4413-a2ff-d2ebe8fe9c71 3 2024-09-16T12:58:34.939 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:34.936+0000 7f6de9003000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T12:58:35.001 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:34.992+0000 7f0e49df3000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T12:58:35.060 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:35.052+0000 7f0e49df3000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T12:58:35.070 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:35.064+0000 7f6de9003000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T12:58:35.111 INFO:teuthology.orchestra.run.smithi136.stdout:3 2024-09-16T12:58:35.123 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 278bfad1-010b-494e-ba61-bdf3935ca078 4 2024-09-16T12:58:35.179 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:35.172+0000 7f0e49df3000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T12:58:35.309 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:35.304+0000 7f6de9003000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T12:58:35.318 INFO:teuthology.orchestra.run.smithi136.stdout:4 2024-09-16T12:58:35.330 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new ed1af73b-b108-4189-8223-f017e634e3e8 5 2024-09-16T12:58:35.362 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:35.356+0000 7f6de9003000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T12:58:35.424 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:35.416+0000 7f0e49df3000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T12:58:35.478 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:35.468+0000 7f0e49df3000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T12:58:35.522 INFO:teuthology.orchestra.run.smithi136.stdout:5 2024-09-16T12:58:35.535 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new ce904ba8-02e9-4579-9635-b4b6c6de1a08 6 2024-09-16T12:58:35.538 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:35.532+0000 7f6de9003000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T12:58:35.597 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:35.592+0000 7f6de9003000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T12:58:35.658 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:35.652+0000 7f6de9003000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T12:58:35.674 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:35.664+0000 7f0e49df3000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T12:58:35.728 INFO:teuthology.orchestra.run.smithi136.stdout:6 2024-09-16T12:58:35.741 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new d911da1b-2725-46a8-bbfa-8968de47271f 7 2024-09-16T12:58:35.742 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:35.736+0000 7f0e49df3000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T12:58:35.810 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:35.800+0000 7f0e49df3000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T12:58:35.929 INFO:teuthology.orchestra.run.smithi136.stdout:7 2024-09-16T12:58:35.946 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T12:58:35.946 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T12:58:35.948 INFO:tasks.ceph.osd.0:Started 2024-09-16T12:58:35.949 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T12:58:35.949 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T12:58:35.951 INFO:tasks.ceph.osd.1:Started 2024-09-16T12:58:35.951 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T12:58:35.951 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T12:58:35.953 INFO:tasks.ceph.osd.2:Started 2024-09-16T12:58:35.953 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T12:58:35.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T12:58:35.955 INFO:tasks.ceph.osd.3:Started 2024-09-16T12:58:35.955 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T12:58:35.955 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T12:58:35.957 INFO:tasks.ceph.osd.4:Started 2024-09-16T12:58:35.958 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T12:58:35.958 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T12:58:35.960 INFO:tasks.ceph.osd.5:Started 2024-09-16T12:58:35.960 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T12:58:35.960 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T12:58:35.962 INFO:tasks.ceph.osd.6:Started 2024-09-16T12:58:35.962 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T12:58:35.962 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T12:58:35.964 INFO:tasks.ceph.osd.7:Started 2024-09-16T12:58:35.965 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T12:58:36.101 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:36.096+0000 7f6de9003000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T12:58:36.158 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:58:36.159 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T12:58:35.921772+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T12:58:35.921772+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T12:58:36.178 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T12:58:36.178 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T12:58:36.212 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:36.208+0000 7f6de9003000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T12:58:36.227 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:36.220+0000 7f0e49df3000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T12:58:36.316 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:58:36.308+0000 7fbba1d1b540 -1 Falling back to public interface 2024-09-16T12:58:36.321 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:58:36.312+0000 7f31d3c22540 -1 Falling back to public interface 2024-09-16T12:58:36.348 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:58:36.344+0000 7fa4b03db540 -1 Falling back to public interface 2024-09-16T12:58:36.356 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:36.348+0000 7f0e49df3000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T12:58:36.505 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:36.500+0000 7f6de9003000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T12:58:36.563 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:58:36.560+0000 7f1930573540 -1 Falling back to public interface 2024-09-16T12:58:36.576 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:58:36.572+0000 7f2dcf042540 -1 Falling back to public interface 2024-09-16T12:58:36.584 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:58:36.580+0000 7f867299b540 -1 Falling back to public interface 2024-09-16T12:58:36.588 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:58:36.580+0000 7f3522109540 -1 Falling back to public interface 2024-09-16T12:58:36.589 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:58:36.580+0000 7fb9bee66540 -1 Falling back to public interface 2024-09-16T12:58:36.668 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:36.664+0000 7f6de9003000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T12:58:36.686 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:36.676+0000 7f0e49df3000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T12:58:36.734 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:36.728+0000 7f6de9003000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T12:58:36.785 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:36.780+0000 7f6de9003000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T12:58:36.842 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:36.840+0000 7f6de9003000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T12:58:36.860 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:36.852+0000 7f0e49df3000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T12:58:36.929 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:36.920+0000 7f0e49df3000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T12:58:36.983 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T12:58:36.983 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:36.976+0000 7f0e49df3000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T12:58:37.028 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:37.024+0000 7f6de9003000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T12:58:37.045 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:37.036+0000 7f0e49df3000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T12:58:37.162 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-16T12:58:37.162 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":10,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T12:58:35.921772+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T12:58:35.921772+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T12:58:37.172 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-16T12:58:37.196 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:37.192+0000 7f6de9003000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T12:58:37.214 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:37.208+0000 7f0e49df3000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T12:58:37.261 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:37.256+0000 7f6de9003000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T12:58:37.391 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:37.384+0000 7f0e49df3000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T12:58:37.420 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:37.416+0000 7f6de9003000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T12:58:37.464 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:37.456+0000 7f0e49df3000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T12:58:37.489 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:37.484+0000 7f6de9003000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T12:58:37.644 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:37.636+0000 7f0e49df3000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T12:58:37.721 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:37.712+0000 7f0e49df3000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T12:58:38.193 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:38.188+0000 7f6de9003000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T12:58:38.483 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-16T12:58:38.475+0000 7f0e49df3000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T12:58:39.235 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:39.232+0000 7f6dd2077700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T12:58:39.239 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-09-16T12:58:39.236+0000 7f6dd2077700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T12:58:40.127 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:58:40.124+0000 7fa4b03db540 -1 osd.7 0 log_to_monitors true 2024-09-16T12:58:40.171 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:58:40.163+0000 7f31d3c22540 -1 osd.1 0 log_to_monitors true 2024-09-16T12:58:40.221 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:58:40.211+0000 7f3522109540 -1 osd.0 0 log_to_monitors true 2024-09-16T12:58:40.235 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:58:40.227+0000 7fbba1d1b540 -1 osd.2 0 log_to_monitors true 2024-09-16T12:58:40.318 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:58:40.316+0000 7f1930573540 -1 osd.5 0 log_to_monitors true 2024-09-16T12:58:40.395 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:58:40.387+0000 7fb9bee66540 -1 osd.3 0 log_to_monitors true 2024-09-16T12:58:40.405 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:58:40.400+0000 7f2dcf042540 -1 osd.4 0 log_to_monitors true 2024-09-16T12:58:40.408 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:58:40.404+0000 7f867299b540 -1 osd.6 0 log_to_monitors true 2024-09-16T12:58:42.530 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:58:42.524+0000 7fa4abb7b700 -1 osd.7 0 waiting for initial osdmap 2024-09-16T12:58:42.536 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:58:42.532+0000 7fa4a491e700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T12:58:42.622 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:58:42.611+0000 7f31cf3c2700 -1 osd.1 0 waiting for initial osdmap 2024-09-16T12:58:42.629 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:58:42.619+0000 7f31c8165700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T12:58:43.527 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:58:43.519+0000 7fbb9d4bb700 -1 osd.2 0 waiting for initial osdmap 2024-09-16T12:58:43.533 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:58:43.523+0000 7fbb9625e700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T12:58:43.537 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:58:43.527+0000 7f351f0ce700 -1 osd.0 0 waiting for initial osdmap 2024-09-16T12:58:43.541 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:58:43.531+0000 7f351664c700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T12:58:43.564 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:58:43.555+0000 7fb9bbe2b700 -1 osd.3 0 waiting for initial osdmap 2024-09-16T12:58:43.569 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:58:43.559+0000 7fb9b33a9700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T12:58:43.576 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:58:43.572+0000 7f866f960700 -1 osd.6 0 waiting for initial osdmap 2024-09-16T12:58:43.577 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:58:43.572+0000 7f192d538700 -1 osd.5 0 waiting for initial osdmap 2024-09-16T12:58:43.581 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:58:43.576+0000 7f8666ede700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T12:58:43.584 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:58:43.580+0000 7f1924ab6700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T12:58:43.586 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:58:43.580+0000 7f2dcc007700 -1 osd.4 0 waiting for initial osdmap 2024-09-16T12:58:43.591 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:58:43.588+0000 7f2dc3585700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T12:58:43.978 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T12:58:44.260 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-16T12:58:44.260 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":14,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T12:58:43.529206+0000","last_up_change":"2024-09-16T12:58:43.529206+0000","last_in_change":"2024-09-16T12:58:35.921772+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":22184},{"type":"v1","addr":"172.21.15.42:6809","nonce":22184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":22184},{"type":"v1","addr":"172.21.15.42:6811","nonce":22184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":22184},{"type":"v1","addr":"172.21.15.42:6815","nonce":22184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":22184},{"type":"v1","addr":"172.21.15.42:6813","nonce":22184}]},"public_addr":"172.21.15.42:6809/22184","cluster_addr":"172.21.15.42:6811/22184","heartbeat_back_addr":"172.21.15.42:6815/22184","heartbeat_front_addr":"172.21.15.42:6813/22184","state":["exists","up"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22341},{"type":"v1","addr":"172.21.15.136:6801","nonce":22341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22341},{"type":"v1","addr":"172.21.15.136:6803","nonce":22341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22341},{"type":"v1","addr":"172.21.15.136:6807","nonce":22341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22341},{"type":"v1","addr":"172.21.15.136:6805","nonce":22341}]},"public_addr":"172.21.15.136:6801/22341","cluster_addr":"172.21.15.136:6803/22341","heartbeat_back_addr":"172.21.15.136:6807/22341","heartbeat_front_addr":"172.21.15.136:6805/22341","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T12:58:44.276 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-09-16T12:58:51.083 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T12:58:51.382 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-16T12:58:51.382 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":17,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T12:58:46.536733+0000","last_up_change":"2024-09-16T12:58:44.530413+0000","last_in_change":"2024-09-16T12:58:35.921772+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T12:58:44.284177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":22186},{"type":"v1","addr":"172.21.15.42:6819","nonce":22186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":22186},{"type":"v1","addr":"172.21.15.42:6823","nonce":22186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":22186},{"type":"v1","addr":"172.21.15.42:6830","nonce":22186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":22186},{"type":"v1","addr":"172.21.15.42:6826","nonce":22186}]},"public_addr":"172.21.15.42:6819/22186","cluster_addr":"172.21.15.42:6823/22186","heartbeat_back_addr":"172.21.15.42:6830/22186","heartbeat_front_addr":"172.21.15.42:6826/22186","state":["exists","up"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":22184},{"type":"v1","addr":"172.21.15.42:6809","nonce":22184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":22184},{"type":"v1","addr":"172.21.15.42:6811","nonce":22184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":22184},{"type":"v1","addr":"172.21.15.42:6815","nonce":22184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":22184},{"type":"v1","addr":"172.21.15.42:6813","nonce":22184}]},"public_addr":"172.21.15.42:6809/22184","cluster_addr":"172.21.15.42:6811/22184","heartbeat_back_addr":"172.21.15.42:6815/22184","heartbeat_front_addr":"172.21.15.42:6813/22184","state":["exists","up"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":22185},{"type":"v1","addr":"172.21.15.42:6801","nonce":22185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":22185},{"type":"v1","addr":"172.21.15.42:6803","nonce":22185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":22185},{"type":"v1","addr":"172.21.15.42:6807","nonce":22185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":22185},{"type":"v1","addr":"172.21.15.42:6805","nonce":22185}]},"public_addr":"172.21.15.42:6801/22185","cluster_addr":"172.21.15.42:6803/22185","heartbeat_back_addr":"172.21.15.42:6807/22185","heartbeat_front_addr":"172.21.15.42:6805/22185","state":["exists","up"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":22187},{"type":"v1","addr":"172.21.15.42:6818","nonce":22187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":22187},{"type":"v1","addr":"172.21.15.42:6822","nonce":22187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":22187},{"type":"v1","addr":"172.21.15.42:6831","nonce":22187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":22187},{"type":"v1","addr":"172.21.15.42:6827","nonce":22187}]},"public_addr":"172.21.15.42:6818/22187","cluster_addr":"172.21.15.42:6822/22187","heartbeat_back_addr":"172.21.15.42:6831/22187","heartbeat_front_addr":"172.21.15.42:6827/22187","state":["exists","up"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22339},{"type":"v1","addr":"172.21.15.136:6817","nonce":22339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":22339},{"type":"v1","addr":"172.21.15.136:6819","nonce":22339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":22339},{"type":"v1","addr":"172.21.15.136:6823","nonce":22339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22339},{"type":"v1","addr":"172.21.15.136:6821","nonce":22339}]},"public_addr":"172.21.15.136:6817/22339","cluster_addr":"172.21.15.136:6819/22339","heartbeat_back_addr":"172.21.15.136:6823/22339","heartbeat_front_addr":"172.21.15.136:6821/22339","state":["exists","up"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22340},{"type":"v1","addr":"172.21.15.136:6809","nonce":22340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22340},{"type":"v1","addr":"172.21.15.136:6811","nonce":22340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22340},{"type":"v1","addr":"172.21.15.136:6815","nonce":22340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22340},{"type":"v1","addr":"172.21.15.136:6813","nonce":22340}]},"public_addr":"172.21.15.136:6809/22340","cluster_addr":"172.21.15.136:6811/22340","heartbeat_back_addr":"172.21.15.136:6815/22340","heartbeat_front_addr":"172.21.15.136:6813/22340","state":["exists","up"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22364},{"type":"v1","addr":"172.21.15.136:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":22364},{"type":"v1","addr":"172.21.15.136:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":22364},{"type":"v1","addr":"172.21.15.136:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22364},{"type":"v1","addr":"172.21.15.136:6829","nonce":22364}]},"public_addr":"172.21.15.136:6825/22364","cluster_addr":"172.21.15.136:6827/22364","heartbeat_back_addr":"172.21.15.136:6831/22364","heartbeat_front_addr":"172.21.15.136:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22341},{"type":"v1","addr":"172.21.15.136:6801","nonce":22341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22341},{"type":"v1","addr":"172.21.15.136:6803","nonce":22341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22341},{"type":"v1","addr":"172.21.15.136:6807","nonce":22341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22341},{"type":"v1","addr":"172.21.15.136:6805","nonce":22341}]},"public_addr":"172.21.15.136:6801/22341","cluster_addr":"172.21.15.136:6803/22341","heartbeat_back_addr":"172.21.15.136:6807/22341","heartbeat_front_addr":"172.21.15.136:6805/22341","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.264463+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.214474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.217927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.435822+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.451286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.287403+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.440296+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.140630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T12:58:51.395 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-16T12:58:51.395 INFO:tasks.ceph:Creating RBD pool 2024-09-16T12:58:51.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T12:58:52.629 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'rbd' created 2024-09-16T12:58:52.642 DEBUG:teuthology.orchestra.run.smithi042:> rbd --cluster ceph pool init rbd 2024-09-16T12:58:55.661 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T12:58:55.662 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T12:58:55.662 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T12:58:55.662 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T12:58:55.662 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T12:58:56.010 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:58:56.024 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":22043},{"type":"v1","addr":"172.21.15.136:6833","nonce":22043}]},"active_addr":"172.21.15.136:6833/22043","active_change":"2024-09-16T12:58:38.209302+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.136:0","nonce":2765937145}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.136:0","nonce":3795352002}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.136:0","nonce":840744221}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.136:0","nonce":2900417850}]}]} 2024-09-16T12:58:56.028 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T12:58:56.028 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T12:58:56.028 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T12:58:56.323 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:58:56.323 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T12:58:55.632996+0000","last_up_change":"2024-09-16T12:58:44.530413+0000","last_in_change":"2024-09-16T12:58:35.921772+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T12:58:44.284177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T12:58:51.733915+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":22186},{"type":"v1","addr":"172.21.15.42:6819","nonce":22186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":22186},{"type":"v1","addr":"172.21.15.42:6823","nonce":22186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":22186},{"type":"v1","addr":"172.21.15.42:6830","nonce":22186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":22186},{"type":"v1","addr":"172.21.15.42:6826","nonce":22186}]},"public_addr":"172.21.15.42:6819/22186","cluster_addr":"172.21.15.42:6823/22186","heartbeat_back_addr":"172.21.15.42:6830/22186","heartbeat_front_addr":"172.21.15.42:6826/22186","state":["exists","up"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":22184},{"type":"v1","addr":"172.21.15.42:6809","nonce":22184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":22184},{"type":"v1","addr":"172.21.15.42:6811","nonce":22184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":22184},{"type":"v1","addr":"172.21.15.42:6815","nonce":22184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":22184},{"type":"v1","addr":"172.21.15.42:6813","nonce":22184}]},"public_addr":"172.21.15.42:6809/22184","cluster_addr":"172.21.15.42:6811/22184","heartbeat_back_addr":"172.21.15.42:6815/22184","heartbeat_front_addr":"172.21.15.42:6813/22184","state":["exists","up"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":22185},{"type":"v1","addr":"172.21.15.42:6801","nonce":22185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":22185},{"type":"v1","addr":"172.21.15.42:6803","nonce":22185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":22185},{"type":"v1","addr":"172.21.15.42:6807","nonce":22185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":22185},{"type":"v1","addr":"172.21.15.42:6805","nonce":22185}]},"public_addr":"172.21.15.42:6801/22185","cluster_addr":"172.21.15.42:6803/22185","heartbeat_back_addr":"172.21.15.42:6807/22185","heartbeat_front_addr":"172.21.15.42:6805/22185","state":["exists","up"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":22187},{"type":"v1","addr":"172.21.15.42:6818","nonce":22187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":22187},{"type":"v1","addr":"172.21.15.42:6822","nonce":22187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":22187},{"type":"v1","addr":"172.21.15.42:6831","nonce":22187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":22187},{"type":"v1","addr":"172.21.15.42:6827","nonce":22187}]},"public_addr":"172.21.15.42:6818/22187","cluster_addr":"172.21.15.42:6822/22187","heartbeat_back_addr":"172.21.15.42:6831/22187","heartbeat_front_addr":"172.21.15.42:6827/22187","state":["exists","up"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22339},{"type":"v1","addr":"172.21.15.136:6817","nonce":22339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":22339},{"type":"v1","addr":"172.21.15.136:6819","nonce":22339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":22339},{"type":"v1","addr":"172.21.15.136:6823","nonce":22339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22339},{"type":"v1","addr":"172.21.15.136:6821","nonce":22339}]},"public_addr":"172.21.15.136:6817/22339","cluster_addr":"172.21.15.136:6819/22339","heartbeat_back_addr":"172.21.15.136:6823/22339","heartbeat_front_addr":"172.21.15.136:6821/22339","state":["exists","up"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22340},{"type":"v1","addr":"172.21.15.136:6809","nonce":22340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22340},{"type":"v1","addr":"172.21.15.136:6811","nonce":22340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22340},{"type":"v1","addr":"172.21.15.136:6815","nonce":22340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22340},{"type":"v1","addr":"172.21.15.136:6813","nonce":22340}]},"public_addr":"172.21.15.136:6809/22340","cluster_addr":"172.21.15.136:6811/22340","heartbeat_back_addr":"172.21.15.136:6815/22340","heartbeat_front_addr":"172.21.15.136:6813/22340","state":["exists","up"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22364},{"type":"v1","addr":"172.21.15.136:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":22364},{"type":"v1","addr":"172.21.15.136:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":22364},{"type":"v1","addr":"172.21.15.136:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22364},{"type":"v1","addr":"172.21.15.136:6829","nonce":22364}]},"public_addr":"172.21.15.136:6825/22364","cluster_addr":"172.21.15.136:6827/22364","heartbeat_back_addr":"172.21.15.136:6831/22364","heartbeat_front_addr":"172.21.15.136:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22341},{"type":"v1","addr":"172.21.15.136:6801","nonce":22341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22341},{"type":"v1","addr":"172.21.15.136:6803","nonce":22341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22341},{"type":"v1","addr":"172.21.15.136:6807","nonce":22341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22341},{"type":"v1","addr":"172.21.15.136:6805","nonce":22341}]},"public_addr":"172.21.15.136:6801/22341","cluster_addr":"172.21.15.136:6803/22341","heartbeat_back_addr":"172.21.15.136:6807/22341","heartbeat_front_addr":"172.21.15.136:6805/22341","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.264463+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.214474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.217927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.435822+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.451286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.287403+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.440296+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.140630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T12:58:56.336 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T12:58:56.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T12:58:56.628 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:58:56.628 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T12:58:55.632996+0000","last_up_change":"2024-09-16T12:58:44.530413+0000","last_in_change":"2024-09-16T12:58:35.921772+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T12:58:44.284177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T12:58:51.733915+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":22186},{"type":"v1","addr":"172.21.15.42:6819","nonce":22186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":22186},{"type":"v1","addr":"172.21.15.42:6823","nonce":22186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":22186},{"type":"v1","addr":"172.21.15.42:6830","nonce":22186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":22186},{"type":"v1","addr":"172.21.15.42:6826","nonce":22186}]},"public_addr":"172.21.15.42:6819/22186","cluster_addr":"172.21.15.42:6823/22186","heartbeat_back_addr":"172.21.15.42:6830/22186","heartbeat_front_addr":"172.21.15.42:6826/22186","state":["exists","up"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":22184},{"type":"v1","addr":"172.21.15.42:6809","nonce":22184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":22184},{"type":"v1","addr":"172.21.15.42:6811","nonce":22184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":22184},{"type":"v1","addr":"172.21.15.42:6815","nonce":22184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":22184},{"type":"v1","addr":"172.21.15.42:6813","nonce":22184}]},"public_addr":"172.21.15.42:6809/22184","cluster_addr":"172.21.15.42:6811/22184","heartbeat_back_addr":"172.21.15.42:6815/22184","heartbeat_front_addr":"172.21.15.42:6813/22184","state":["exists","up"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":22185},{"type":"v1","addr":"172.21.15.42:6801","nonce":22185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":22185},{"type":"v1","addr":"172.21.15.42:6803","nonce":22185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":22185},{"type":"v1","addr":"172.21.15.42:6807","nonce":22185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":22185},{"type":"v1","addr":"172.21.15.42:6805","nonce":22185}]},"public_addr":"172.21.15.42:6801/22185","cluster_addr":"172.21.15.42:6803/22185","heartbeat_back_addr":"172.21.15.42:6807/22185","heartbeat_front_addr":"172.21.15.42:6805/22185","state":["exists","up"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":22187},{"type":"v1","addr":"172.21.15.42:6818","nonce":22187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":22187},{"type":"v1","addr":"172.21.15.42:6822","nonce":22187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":22187},{"type":"v1","addr":"172.21.15.42:6831","nonce":22187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":22187},{"type":"v1","addr":"172.21.15.42:6827","nonce":22187}]},"public_addr":"172.21.15.42:6818/22187","cluster_addr":"172.21.15.42:6822/22187","heartbeat_back_addr":"172.21.15.42:6831/22187","heartbeat_front_addr":"172.21.15.42:6827/22187","state":["exists","up"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22339},{"type":"v1","addr":"172.21.15.136:6817","nonce":22339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":22339},{"type":"v1","addr":"172.21.15.136:6819","nonce":22339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":22339},{"type":"v1","addr":"172.21.15.136:6823","nonce":22339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22339},{"type":"v1","addr":"172.21.15.136:6821","nonce":22339}]},"public_addr":"172.21.15.136:6817/22339","cluster_addr":"172.21.15.136:6819/22339","heartbeat_back_addr":"172.21.15.136:6823/22339","heartbeat_front_addr":"172.21.15.136:6821/22339","state":["exists","up"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22340},{"type":"v1","addr":"172.21.15.136:6809","nonce":22340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22340},{"type":"v1","addr":"172.21.15.136:6811","nonce":22340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22340},{"type":"v1","addr":"172.21.15.136:6815","nonce":22340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22340},{"type":"v1","addr":"172.21.15.136:6813","nonce":22340}]},"public_addr":"172.21.15.136:6809/22340","cluster_addr":"172.21.15.136:6811/22340","heartbeat_back_addr":"172.21.15.136:6815/22340","heartbeat_front_addr":"172.21.15.136:6813/22340","state":["exists","up"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22364},{"type":"v1","addr":"172.21.15.136:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":22364},{"type":"v1","addr":"172.21.15.136:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":22364},{"type":"v1","addr":"172.21.15.136:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22364},{"type":"v1","addr":"172.21.15.136:6829","nonce":22364}]},"public_addr":"172.21.15.136:6825/22364","cluster_addr":"172.21.15.136:6827/22364","heartbeat_back_addr":"172.21.15.136:6831/22364","heartbeat_front_addr":"172.21.15.136:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22341},{"type":"v1","addr":"172.21.15.136:6801","nonce":22341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22341},{"type":"v1","addr":"172.21.15.136:6803","nonce":22341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22341},{"type":"v1","addr":"172.21.15.136:6807","nonce":22341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22341},{"type":"v1","addr":"172.21.15.136:6805","nonce":22341}]},"public_addr":"172.21.15.136:6801/22341","cluster_addr":"172.21.15.136:6803/22341","heartbeat_back_addr":"172.21.15.136:6807/22341","heartbeat_front_addr":"172.21.15.136:6805/22341","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.264463+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.214474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.217927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.435822+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.451286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.287403+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.440296+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.140630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T12:58:56.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T12:58:56.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T12:58:56.643 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T12:58:56.643 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T12:58:56.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T12:58:56.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T12:58:56.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T12:58:56.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T12:58:56.841 INFO:teuthology.orchestra.run.smithi042.stdout:60129542149 2024-09-16T12:58:56.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T12:58:56.842 INFO:teuthology.orchestra.run.smithi042.stdout:64424509447 2024-09-16T12:58:56.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T12:58:56.857 INFO:teuthology.orchestra.run.smithi042.stdout:60129542150 2024-09-16T12:58:56.857 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T12:58:56.858 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-16T12:58:56.858 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T12:58:56.868 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-16T12:58:56.869 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T12:58:56.870 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-16T12:58:56.871 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T12:58:56.874 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-16T12:58:56.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T12:58:56.878 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-16T12:58:56.879 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T12:58:57.307 INFO:teuthology.orchestra.run.smithi042.stdout:60129542148 2024-09-16T12:58:57.328 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.1 2024-09-16T12:58:57.337 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-16T12:58:57.350 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-16T12:58:57.350 INFO:teuthology.orchestra.run.smithi042.stdout:60129542149 2024-09-16T12:58:57.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.0 2024-09-16T12:58:57.362 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-16T12:58:57.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2024-09-16T12:58:57.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.6 2024-09-16T12:58:57.370 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-16T12:58:57.370 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-16T12:58:57.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-09-16T12:58:57.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.5 2024-09-16T12:58:57.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.4 2024-09-16T12:58:57.395 INFO:teuthology.orchestra.run.smithi042.stdout:64424509443 2024-09-16T12:58:57.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.3 2024-09-16T12:58:58.329 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T12:58:58.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T12:58:58.366 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T12:58:58.366 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T12:58:58.377 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T12:58:58.383 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T12:58:58.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T12:58:58.408 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T12:58:58.782 INFO:teuthology.orchestra.run.smithi042.stdout:60129542149 2024-09-16T12:58:58.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2024-09-16T12:58:58.802 DEBUG:teuthology.parallel:result is None 2024-09-16T12:58:58.822 INFO:teuthology.orchestra.run.smithi042.stdout:64424509447 2024-09-16T12:58:58.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-09-16T12:58:58.840 DEBUG:teuthology.parallel:result is None 2024-09-16T12:58:58.863 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-16T12:58:58.870 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-16T12:58:58.874 INFO:teuthology.orchestra.run.smithi042.stdout:60129542150 2024-09-16T12:58:58.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-09-16T12:58:58.875 DEBUG:teuthology.parallel:result is None 2024-09-16T12:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-16T12:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-16T12:58:58.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.3 2024-09-16T12:58:58.885 DEBUG:teuthology.parallel:result is None 2024-09-16T12:58:58.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2024-09-16T12:58:58.889 DEBUG:teuthology.parallel:result is None 2024-09-16T12:58:58.895 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.6 2024-09-16T12:58:58.896 DEBUG:teuthology.parallel:result is None 2024-09-16T12:58:58.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.4 2024-09-16T12:58:58.896 DEBUG:teuthology.parallel:result is None 2024-09-16T12:58:58.899 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-16T12:58:58.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.5 2024-09-16T12:58:58.911 DEBUG:teuthology.parallel:result is None 2024-09-16T12:58:58.911 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T12:58:58.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T12:58:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:58:59.235 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T12:58:59.245 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-16T12:58:58.663591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2401392,"kb_used_data":1728,"kb_used_omap":13,"kb_used_meta":2399538,"kb_avail":752573328,"statfs":{"total":773094113280,"available":770635087872,"internally_reserved":0,"allocated":1769472,"data_stored":1728010,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2457127616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.042586"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:56.031300+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T12:58:56.031300+0000","last_peered":"2024-09-16T12:58:56.031300+0000","last_clean":"2024-09-16T12:58:56.031300+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T12:58:56.031300+0000","last_undegraded":"2024-09-16T12:58:56.031300+0000","last_fullsized":"2024-09-16T12:58:56.031300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:55.653321+0000","last_change":"2024-09-16T12:58:55.653750+0000","last_active":"2024-09-16T12:58:55.653321+0000","last_peered":"2024-09-16T12:58:55.653321+0000","last_clean":"2024-09-16T12:58:55.653321+0000","last_became_active":"2024-09-16T12:58:53.634053+0000","last_became_peered":"2024-09-16T12:58:53.634053+0000","last_unstale":"2024-09-16T12:58:55.653321+0000","last_undegraded":"2024-09-16T12:58:55.653321+0000","last_fullsized":"2024-09-16T12:58:55.653321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:50.402394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065616299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:56.031391+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T12:58:56.031391+0000","last_peered":"2024-09-16T12:58:56.031391+0000","last_clean":"2024-09-16T12:58:56.031391+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T12:58:56.031391+0000","last_undegraded":"2024-09-16T12:58:56.031391+0000","last_fullsized":"2024-09-16T12:58:56.031391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:55.653381+0000","last_change":"2024-09-16T12:58:55.653753+0000","last_active":"2024-09-16T12:58:55.653381+0000","last_peered":"2024-09-16T12:58:55.653381+0000","last_clean":"2024-09-16T12:58:55.653381+0000","last_became_active":"2024-09-16T12:58:53.630727+0000","last_became_peered":"2024-09-16T12:58:53.630727+0000","last_unstale":"2024-09-16T12:58:55.653381+0000","last_undegraded":"2024-09-16T12:58:55.653381+0000","last_fullsized":"2024-09-16T12:58:55.653381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:00.855732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061053599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:55.654478+0000","last_change":"2024-09-16T12:58:55.654478+0000","last_active":"2024-09-16T12:58:55.654478+0000","last_peered":"2024-09-16T12:58:55.654478+0000","last_clean":"2024-09-16T12:58:55.654478+0000","last_became_active":"2024-09-16T12:58:53.631689+0000","last_became_peered":"2024-09-16T12:58:53.631689+0000","last_unstale":"2024-09-16T12:58:55.654478+0000","last_undegraded":"2024-09-16T12:58:55.654478+0000","last_fullsized":"2024-09-16T12:58:55.654478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:28.863018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:55.646178+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T12:58:55.646178+0000","last_peered":"2024-09-16T12:58:55.646178+0000","last_clean":"2024-09-16T12:58:55.646178+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T12:58:55.646178+0000","last_undegraded":"2024-09-16T12:58:55.646178+0000","last_fullsized":"2024-09-16T12:58:55.646178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:56.031266+0000","last_change":"2024-09-16T12:58:56.031861+0000","last_active":"2024-09-16T12:58:56.031266+0000","last_peered":"2024-09-16T12:58:56.031266+0000","last_clean":"2024-09-16T12:58:56.031266+0000","last_became_active":"2024-09-16T12:58:53.635856+0000","last_became_peered":"2024-09-16T12:58:53.635856+0000","last_unstale":"2024-09-16T12:58:56.031266+0000","last_undegraded":"2024-09-16T12:58:56.031266+0000","last_fullsized":"2024-09-16T12:58:56.031266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:40.827808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:55.652178+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T12:58:55.652178+0000","last_peered":"2024-09-16T12:58:55.652178+0000","last_clean":"2024-09-16T12:58:55.652178+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T12:58:55.652178+0000","last_undegraded":"2024-09-16T12:58:55.652178+0000","last_fullsized":"2024-09-16T12:58:55.652178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":102,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T12:58:53.630016+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T12:58:53.630016+0000","last_peered":"2024-09-16T12:58:53.630016+0000","last_clean":"2024-09-16T12:58:53.630016+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T12:58:53.630016+0000","last_undegraded":"2024-09-16T12:58:53.630016+0000","last_fullsized":"2024-09-16T12:58:53.630016+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":69849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":212992,"data_stored":69868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.736}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297564,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":208896,"data_stored":69849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]}]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301724,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070116,"statfs":{"total":96636764160,"available":96327798784,"internally_reserved":0,"allocated":208896,"data_stored":69849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301720,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070120,"statfs":{"total":96636764160,"available":96327802880,"internally_reserved":0,"allocated":208896,"data_stored":69849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":266240,"data_stored":660217,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.593}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]}]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301756,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070084,"statfs":{"total":96636764160,"available":96327766016,"internally_reserved":0,"allocated":241664,"data_stored":648661,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301728,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070112,"statfs":{"total":96636764160,"available":96327794688,"internally_reserved":0,"allocated":212992,"data_stored":69868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T12:58:59.246 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T12:58:59.525 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:58:59.528 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T12:58:59.538 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-16T12:58:58.663591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2401392,"kb_used_data":1728,"kb_used_omap":13,"kb_used_meta":2399538,"kb_avail":752573328,"statfs":{"total":773094113280,"available":770635087872,"internally_reserved":0,"allocated":1769472,"data_stored":1728010,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2457127616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.042586"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:56.031300+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T12:58:56.031300+0000","last_peered":"2024-09-16T12:58:56.031300+0000","last_clean":"2024-09-16T12:58:56.031300+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T12:58:56.031300+0000","last_undegraded":"2024-09-16T12:58:56.031300+0000","last_fullsized":"2024-09-16T12:58:56.031300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:55.653321+0000","last_change":"2024-09-16T12:58:55.653750+0000","last_active":"2024-09-16T12:58:55.653321+0000","last_peered":"2024-09-16T12:58:55.653321+0000","last_clean":"2024-09-16T12:58:55.653321+0000","last_became_active":"2024-09-16T12:58:53.634053+0000","last_became_peered":"2024-09-16T12:58:53.634053+0000","last_unstale":"2024-09-16T12:58:55.653321+0000","last_undegraded":"2024-09-16T12:58:55.653321+0000","last_fullsized":"2024-09-16T12:58:55.653321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:50.402394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065616299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:56.031391+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T12:58:56.031391+0000","last_peered":"2024-09-16T12:58:56.031391+0000","last_clean":"2024-09-16T12:58:56.031391+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T12:58:56.031391+0000","last_undegraded":"2024-09-16T12:58:56.031391+0000","last_fullsized":"2024-09-16T12:58:56.031391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:55.653381+0000","last_change":"2024-09-16T12:58:55.653753+0000","last_active":"2024-09-16T12:58:55.653381+0000","last_peered":"2024-09-16T12:58:55.653381+0000","last_clean":"2024-09-16T12:58:55.653381+0000","last_became_active":"2024-09-16T12:58:53.630727+0000","last_became_peered":"2024-09-16T12:58:53.630727+0000","last_unstale":"2024-09-16T12:58:55.653381+0000","last_undegraded":"2024-09-16T12:58:55.653381+0000","last_fullsized":"2024-09-16T12:58:55.653381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:00.855732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061053599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:55.654478+0000","last_change":"2024-09-16T12:58:55.654478+0000","last_active":"2024-09-16T12:58:55.654478+0000","last_peered":"2024-09-16T12:58:55.654478+0000","last_clean":"2024-09-16T12:58:55.654478+0000","last_became_active":"2024-09-16T12:58:53.631689+0000","last_became_peered":"2024-09-16T12:58:53.631689+0000","last_unstale":"2024-09-16T12:58:55.654478+0000","last_undegraded":"2024-09-16T12:58:55.654478+0000","last_fullsized":"2024-09-16T12:58:55.654478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:28.863018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:55.646178+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T12:58:55.646178+0000","last_peered":"2024-09-16T12:58:55.646178+0000","last_clean":"2024-09-16T12:58:55.646178+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T12:58:55.646178+0000","last_undegraded":"2024-09-16T12:58:55.646178+0000","last_fullsized":"2024-09-16T12:58:55.646178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:56.031266+0000","last_change":"2024-09-16T12:58:56.031861+0000","last_active":"2024-09-16T12:58:56.031266+0000","last_peered":"2024-09-16T12:58:56.031266+0000","last_clean":"2024-09-16T12:58:56.031266+0000","last_became_active":"2024-09-16T12:58:53.635856+0000","last_became_peered":"2024-09-16T12:58:53.635856+0000","last_unstale":"2024-09-16T12:58:56.031266+0000","last_undegraded":"2024-09-16T12:58:56.031266+0000","last_fullsized":"2024-09-16T12:58:56.031266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:40.827808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T12:58:55.652178+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T12:58:55.652178+0000","last_peered":"2024-09-16T12:58:55.652178+0000","last_clean":"2024-09-16T12:58:55.652178+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T12:58:55.652178+0000","last_undegraded":"2024-09-16T12:58:55.652178+0000","last_fullsized":"2024-09-16T12:58:55.652178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":102,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T12:58:53.630016+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T12:58:53.630016+0000","last_peered":"2024-09-16T12:58:53.630016+0000","last_clean":"2024-09-16T12:58:53.630016+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T12:58:53.630016+0000","last_undegraded":"2024-09-16T12:58:53.630016+0000","last_fullsized":"2024-09-16T12:58:53.630016+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":69849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":212992,"data_stored":69868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.736}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297564,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":208896,"data_stored":69849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]}]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301724,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070116,"statfs":{"total":96636764160,"available":96327798784,"internally_reserved":0,"allocated":208896,"data_stored":69849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301720,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070120,"statfs":{"total":96636764160,"available":96327802880,"internally_reserved":0,"allocated":208896,"data_stored":69849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":266240,"data_stored":660217,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.593}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]}]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301756,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070084,"statfs":{"total":96636764160,"available":96327766016,"internally_reserved":0,"allocated":241664,"data_stored":648661,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301728,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070112,"statfs":{"total":96636764160,"available":96327794688,"internally_reserved":0,"allocated":212992,"data_stored":69868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T12:58:59.539 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T12:58:59.539 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T12:58:59.539 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T12:58:59.540 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T12:58:59.840 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:58:59.840 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T12:58:59.852 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T12:58:59.852 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T12:58:59.863 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T12:58:59.863 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T12:58:59.863 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create cache 4' 2024-09-16T12:59:00.686 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cache' created 2024-09-16T12:59:00.699 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier add rbd cache' 2024-09-16T12:59:01.692 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cache' is now (or already was) a tier of 'rbd' 2024-09-16T12:59:01.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier cache-mode cache writeback' 2024-09-16T12:59:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:set cache-mode for pool 'cache' to writeback 2024-09-16T12:59:02.718 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier set-overlay rbd cache' 2024-09-16T12:59:03.707 INFO:teuthology.orchestra.run.smithi042.stderr:overlay for 'rbd' is now (or already was) 'cache' 2024-09-16T12:59:03.719 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_type bloom' 2024-09-16T12:59:04.715 INFO:teuthology.orchestra.run.smithi042.stderr:set pool 3 hit_set_type to bloom 2024-09-16T12:59:04.729 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_count 8' 2024-09-16T12:59:05.722 INFO:teuthology.orchestra.run.smithi042.stderr:set pool 3 hit_set_count to 8 2024-09-16T12:59:05.735 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_period 60' 2024-09-16T12:59:06.734 INFO:teuthology.orchestra.run.smithi042.stderr:set pool 3 hit_set_period to 60 2024-09-16T12:59:06.748 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache target_max_objects 250' 2024-09-16T12:59:07.742 INFO:teuthology.orchestra.run.smithi042.stderr:set pool 3 target_max_objects to 250 2024-09-16T12:59:07.757 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-16T12:59:07.768 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T12:59:07.768 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-16T12:59:07.768 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-16T12:59:07.768 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-16T12:59:08.093 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-16T12:59:08.275 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T12:59:08.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T12:59:08.586 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:59:08.590 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T12:59:08.601 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-16T12:59:08.230132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2406092,"kb_used_data":2268,"kb_used_omap":13,"kb_used_meta":2403698,"kb_avail":752568628,"statfs":{"total":773094113280,"available":770630275072,"internally_reserved":0,"allocated":2322432,"data_stored":2002667,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2461387456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001058"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.706456+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T12:59:03.706456+0000","last_peered":"2024-09-16T12:59:03.706456+0000","last_clean":"2024-09-16T12:59:03.706456+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T12:59:03.706456+0000","last_undegraded":"2024-09-16T12:59:03.706456+0000","last_fullsized":"2024-09-16T12:59:03.706456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T12:59:01.683881+0000","last_change":"2024-09-16T12:58:55.653750+0000","last_active":"2024-09-16T12:59:01.683881+0000","last_peered":"2024-09-16T12:59:01.683881+0000","last_clean":"2024-09-16T12:59:01.683881+0000","last_became_active":"2024-09-16T12:58:53.634053+0000","last_became_peered":"2024-09-16T12:58:53.634053+0000","last_unstale":"2024-09-16T12:59:01.683881+0000","last_undegraded":"2024-09-16T12:59:01.683881+0000","last_fullsized":"2024-09-16T12:59:01.683881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:50.402394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065616299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.436167+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T12:59:03.436167+0000","last_peered":"2024-09-16T12:59:03.436167+0000","last_clean":"2024-09-16T12:59:03.436167+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T12:59:03.436167+0000","last_undegraded":"2024-09-16T12:59:03.436167+0000","last_fullsized":"2024-09-16T12:59:03.436167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T12:59:01.683849+0000","last_change":"2024-09-16T12:58:55.653753+0000","last_active":"2024-09-16T12:59:01.683849+0000","last_peered":"2024-09-16T12:59:01.683849+0000","last_clean":"2024-09-16T12:59:01.683849+0000","last_became_active":"2024-09-16T12:58:53.630727+0000","last_became_peered":"2024-09-16T12:58:53.630727+0000","last_unstale":"2024-09-16T12:59:01.683849+0000","last_undegraded":"2024-09-16T12:59:01.683849+0000","last_fullsized":"2024-09-16T12:59:01.683849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:00.855732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061053599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.699578+0000","last_change":"2024-09-16T12:59:01.688552+0000","last_active":"2024-09-16T12:59:03.699578+0000","last_peered":"2024-09-16T12:59:03.699578+0000","last_clean":"2024-09-16T12:59:03.699578+0000","last_became_active":"2024-09-16T12:59:01.688068+0000","last_became_peered":"2024-09-16T12:59:01.688068+0000","last_unstale":"2024-09-16T12:59:03.699578+0000","last_undegraded":"2024-09-16T12:59:03.699578+0000","last_fullsized":"2024-09-16T12:59:03.699578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:01.298345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.706316+0000","last_change":"2024-09-16T12:58:55.654478+0000","last_active":"2024-09-16T12:59:03.706316+0000","last_peered":"2024-09-16T12:59:03.706316+0000","last_clean":"2024-09-16T12:59:03.706316+0000","last_became_active":"2024-09-16T12:58:53.631689+0000","last_became_peered":"2024-09-16T12:58:53.631689+0000","last_unstale":"2024-09-16T12:59:03.706316+0000","last_undegraded":"2024-09-16T12:59:03.706316+0000","last_fullsized":"2024-09-16T12:59:03.706316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:28.863018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T12:59:01.686627+0000","last_change":"2024-09-16T12:59:01.685865+0000","last_active":"2024-09-16T12:59:01.686627+0000","last_peered":"2024-09-16T12:59:01.686627+0000","last_clean":"2024-09-16T12:59:01.686627+0000","last_became_active":"2024-09-16T12:59:01.685545+0000","last_became_peered":"2024-09-16T12:59:01.685545+0000","last_unstale":"2024-09-16T12:59:01.686627+0000","last_undegraded":"2024-09-16T12:59:01.686627+0000","last_fullsized":"2024-09-16T12:59:01.686627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:18.484667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.696450+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T12:59:03.696450+0000","last_peered":"2024-09-16T12:59:03.696450+0000","last_clean":"2024-09-16T12:59:03.696450+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T12:59:03.696450+0000","last_undegraded":"2024-09-16T12:59:03.696450+0000","last_fullsized":"2024-09-16T12:59:03.696450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.702648+0000","last_change":"2024-09-16T12:59:01.686741+0000","last_active":"2024-09-16T12:59:03.702648+0000","last_peered":"2024-09-16T12:59:03.702648+0000","last_clean":"2024-09-16T12:59:03.702648+0000","last_became_active":"2024-09-16T12:59:01.686267+0000","last_became_peered":"2024-09-16T12:59:01.686267+0000","last_unstale":"2024-09-16T12:59:03.702648+0000","last_undegraded":"2024-09-16T12:59:03.702648+0000","last_fullsized":"2024-09-16T12:59:03.702648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:49.310418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.436082+0000","last_change":"2024-09-16T12:58:56.031861+0000","last_active":"2024-09-16T12:59:03.436082+0000","last_peered":"2024-09-16T12:59:03.436082+0000","last_clean":"2024-09-16T12:59:03.436082+0000","last_became_active":"2024-09-16T12:58:53.635856+0000","last_became_peered":"2024-09-16T12:58:53.635856+0000","last_unstale":"2024-09-16T12:59:03.436082+0000","last_undegraded":"2024-09-16T12:59:03.436082+0000","last_fullsized":"2024-09-16T12:59:03.436082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:40.827808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.697217+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T12:59:03.697217+0000","last_peered":"2024-09-16T12:59:03.697217+0000","last_clean":"2024-09-16T12:59:03.697217+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T12:59:03.697217+0000","last_undegraded":"2024-09-16T12:59:03.697217+0000","last_fullsized":"2024-09-16T12:59:03.697217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:20.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.706274+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T12:59:03.706274+0000","last_peered":"2024-09-16T12:59:03.706274+0000","last_clean":"2024-09-16T12:59:03.706274+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T12:59:03.706274+0000","last_undegraded":"2024-09-16T12:59:03.706274+0000","last_fullsized":"2024-09-16T12:59:03.706274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":107,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.436134+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T12:59:03.436134+0000","last_peered":"2024-09-16T12:59:03.436134+0000","last_clean":"2024-09-16T12:59:03.436134+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T12:59:03.436134+0000","last_undegraded":"2024-09-16T12:59:03.436134+0000","last_fullsized":"2024-09-16T12:59:03.436134+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297620,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074220,"statfs":{"total":96636764160,"available":96332001280,"internally_reserved":0,"allocated":270336,"data_stored":100430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297628,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074212,"statfs":{"total":96636764160,"available":96331993088,"internally_reserved":0,"allocated":274432,"data_stored":100449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301780,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070060,"statfs":{"total":96636764160,"available":96327741440,"internally_reserved":0,"allocated":270336,"data_stored":100430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.583}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]}]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301796,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070044,"statfs":{"total":96636764160,"available":96327725056,"internally_reserved":0,"allocated":282624,"data_stored":106581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301792,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070048,"statfs":{"total":96636764160,"available":96327729152,"internally_reserved":0,"allocated":282624,"data_stored":106581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":339968,"data_stored":696949,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301840,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070000,"statfs":{"total":96636764160,"available":96327680000,"internally_reserved":0,"allocated":327680,"data_stored":690798,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301784,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070056,"statfs":{"total":96636764160,"available":96327737344,"internally_reserved":0,"allocated":274432,"data_stored":100449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T12:59:08.603 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T12:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:59:08.904 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T12:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-16T12:59:08.230132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2406092,"kb_used_data":2268,"kb_used_omap":13,"kb_used_meta":2403698,"kb_avail":752568628,"statfs":{"total":773094113280,"available":770630275072,"internally_reserved":0,"allocated":2322432,"data_stored":2002667,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2461387456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001058"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.706456+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T12:59:03.706456+0000","last_peered":"2024-09-16T12:59:03.706456+0000","last_clean":"2024-09-16T12:59:03.706456+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T12:59:03.706456+0000","last_undegraded":"2024-09-16T12:59:03.706456+0000","last_fullsized":"2024-09-16T12:59:03.706456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T12:59:01.683881+0000","last_change":"2024-09-16T12:58:55.653750+0000","last_active":"2024-09-16T12:59:01.683881+0000","last_peered":"2024-09-16T12:59:01.683881+0000","last_clean":"2024-09-16T12:59:01.683881+0000","last_became_active":"2024-09-16T12:58:53.634053+0000","last_became_peered":"2024-09-16T12:58:53.634053+0000","last_unstale":"2024-09-16T12:59:01.683881+0000","last_undegraded":"2024-09-16T12:59:01.683881+0000","last_fullsized":"2024-09-16T12:59:01.683881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:50.402394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065616299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.436167+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T12:59:03.436167+0000","last_peered":"2024-09-16T12:59:03.436167+0000","last_clean":"2024-09-16T12:59:03.436167+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T12:59:03.436167+0000","last_undegraded":"2024-09-16T12:59:03.436167+0000","last_fullsized":"2024-09-16T12:59:03.436167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T12:59:01.683849+0000","last_change":"2024-09-16T12:58:55.653753+0000","last_active":"2024-09-16T12:59:01.683849+0000","last_peered":"2024-09-16T12:59:01.683849+0000","last_clean":"2024-09-16T12:59:01.683849+0000","last_became_active":"2024-09-16T12:58:53.630727+0000","last_became_peered":"2024-09-16T12:58:53.630727+0000","last_unstale":"2024-09-16T12:59:01.683849+0000","last_undegraded":"2024-09-16T12:59:01.683849+0000","last_fullsized":"2024-09-16T12:59:01.683849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:00.855732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061053599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.699578+0000","last_change":"2024-09-16T12:59:01.688552+0000","last_active":"2024-09-16T12:59:03.699578+0000","last_peered":"2024-09-16T12:59:03.699578+0000","last_clean":"2024-09-16T12:59:03.699578+0000","last_became_active":"2024-09-16T12:59:01.688068+0000","last_became_peered":"2024-09-16T12:59:01.688068+0000","last_unstale":"2024-09-16T12:59:03.699578+0000","last_undegraded":"2024-09-16T12:59:03.699578+0000","last_fullsized":"2024-09-16T12:59:03.699578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:01.298345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.706316+0000","last_change":"2024-09-16T12:58:55.654478+0000","last_active":"2024-09-16T12:59:03.706316+0000","last_peered":"2024-09-16T12:59:03.706316+0000","last_clean":"2024-09-16T12:59:03.706316+0000","last_became_active":"2024-09-16T12:58:53.631689+0000","last_became_peered":"2024-09-16T12:58:53.631689+0000","last_unstale":"2024-09-16T12:59:03.706316+0000","last_undegraded":"2024-09-16T12:59:03.706316+0000","last_fullsized":"2024-09-16T12:59:03.706316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:28.863018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T12:59:01.686627+0000","last_change":"2024-09-16T12:59:01.685865+0000","last_active":"2024-09-16T12:59:01.686627+0000","last_peered":"2024-09-16T12:59:01.686627+0000","last_clean":"2024-09-16T12:59:01.686627+0000","last_became_active":"2024-09-16T12:59:01.685545+0000","last_became_peered":"2024-09-16T12:59:01.685545+0000","last_unstale":"2024-09-16T12:59:01.686627+0000","last_undegraded":"2024-09-16T12:59:01.686627+0000","last_fullsized":"2024-09-16T12:59:01.686627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:18.484667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.696450+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T12:59:03.696450+0000","last_peered":"2024-09-16T12:59:03.696450+0000","last_clean":"2024-09-16T12:59:03.696450+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T12:59:03.696450+0000","last_undegraded":"2024-09-16T12:59:03.696450+0000","last_fullsized":"2024-09-16T12:59:03.696450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.702648+0000","last_change":"2024-09-16T12:59:01.686741+0000","last_active":"2024-09-16T12:59:03.702648+0000","last_peered":"2024-09-16T12:59:03.702648+0000","last_clean":"2024-09-16T12:59:03.702648+0000","last_became_active":"2024-09-16T12:59:01.686267+0000","last_became_peered":"2024-09-16T12:59:01.686267+0000","last_unstale":"2024-09-16T12:59:03.702648+0000","last_undegraded":"2024-09-16T12:59:03.702648+0000","last_fullsized":"2024-09-16T12:59:03.702648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:49.310418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.436082+0000","last_change":"2024-09-16T12:58:56.031861+0000","last_active":"2024-09-16T12:59:03.436082+0000","last_peered":"2024-09-16T12:59:03.436082+0000","last_clean":"2024-09-16T12:59:03.436082+0000","last_became_active":"2024-09-16T12:58:53.635856+0000","last_became_peered":"2024-09-16T12:58:53.635856+0000","last_unstale":"2024-09-16T12:59:03.436082+0000","last_undegraded":"2024-09-16T12:59:03.436082+0000","last_fullsized":"2024-09-16T12:59:03.436082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:40.827808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.697217+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T12:59:03.697217+0000","last_peered":"2024-09-16T12:59:03.697217+0000","last_clean":"2024-09-16T12:59:03.697217+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T12:59:03.697217+0000","last_undegraded":"2024-09-16T12:59:03.697217+0000","last_fullsized":"2024-09-16T12:59:03.697217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:20.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.706274+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T12:59:03.706274+0000","last_peered":"2024-09-16T12:59:03.706274+0000","last_clean":"2024-09-16T12:59:03.706274+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T12:59:03.706274+0000","last_undegraded":"2024-09-16T12:59:03.706274+0000","last_fullsized":"2024-09-16T12:59:03.706274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":107,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T12:59:03.436134+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T12:59:03.436134+0000","last_peered":"2024-09-16T12:59:03.436134+0000","last_clean":"2024-09-16T12:59:03.436134+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T12:59:03.436134+0000","last_undegraded":"2024-09-16T12:59:03.436134+0000","last_fullsized":"2024-09-16T12:59:03.436134+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297620,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074220,"statfs":{"total":96636764160,"available":96332001280,"internally_reserved":0,"allocated":270336,"data_stored":100430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297628,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074212,"statfs":{"total":96636764160,"available":96331993088,"internally_reserved":0,"allocated":274432,"data_stored":100449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301780,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070060,"statfs":{"total":96636764160,"available":96327741440,"internally_reserved":0,"allocated":270336,"data_stored":100430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.583}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]}]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301796,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070044,"statfs":{"total":96636764160,"available":96327725056,"internally_reserved":0,"allocated":282624,"data_stored":106581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301792,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070048,"statfs":{"total":96636764160,"available":96327729152,"internally_reserved":0,"allocated":282624,"data_stored":106581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301852,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069988,"statfs":{"total":96636764160,"available":96327667712,"internally_reserved":0,"allocated":339968,"data_stored":696949,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301840,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070000,"statfs":{"total":96636764160,"available":96327680000,"internally_reserved":0,"allocated":327680,"data_stored":690798,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301784,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070056,"statfs":{"total":96636764160,"available":96327737344,"internally_reserved":0,"allocated":274432,"data_stored":100449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T12:59:08.918 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T12:59:08.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-16T12:59:09.212 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 30 2024-09-16T12:59:09.212 INFO:teuthology.orchestra.run.smithi042.stdout:fsid c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7 2024-09-16T12:59:09.212 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T12:58:33.966359+0000 2024-09-16T12:59:09.212 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T12:59:07.717929+0000 2024-09-16T12:59:09.212 INFO:teuthology.orchestra.run.smithi042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T12:59:09.212 INFO:teuthology.orchestra.run.smithi042.stdout:crush_version 5 2024-09-16T12:59:09.212 INFO:teuthology.orchestra.run.smithi042.stdout:full_ratio 0.95 2024-09-16T12:59:09.212 INFO:teuthology.orchestra.run.smithi042.stdout:backfillfull_ratio 0.9 2024-09-16T12:59:09.212 INFO:teuthology.orchestra.run.smithi042.stdout:nearfull_ratio 0.85 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:require_min_compat_client luminous 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:min_compat_client jewel 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:require_osd_release quincy 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:stretch_mode_enabled false 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 26 lfor 26/26/26 flags hashpspool,selfmanaged_snaps tiers 3 read_tier 3 write_tier 3 stripe_width 0 application rbd 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:pool 3 'cache' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 4 pgp_num 4 autoscale_mode off last_change 30 lfor 26/26/26 flags hashpspool,incomplete_clones,selfmanaged_snaps tier_of 2 cache_mode writeback target_objects 250 hit_set bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 60s x8 decay_rate 0 search_last_n 0 stripe_width 0 application rbd 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:max_osd 8 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 up in weight 1 up_from 15 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6817/22186,v1:172.21.15.42:6819/22186] [v2:172.21.15.42:6821/22186,v1:172.21.15.42:6823/22186] exists,up 0d8a8277-07cd-40f1-8c98-98b8b51d1689 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6808/22184,v1:172.21.15.42:6809/22184] [v2:172.21.15.42:6810/22184,v1:172.21.15.42:6811/22184] exists,up 6ce5996a-5e0e-4c9d-b268-27b34a14d830 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6800/22185,v1:172.21.15.42:6801/22185] [v2:172.21.15.42:6802/22185,v1:172.21.15.42:6803/22185] exists,up 156042ab-909e-4368-b510-84018cb4354f 2024-09-16T12:59:09.213 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 up in weight 1 up_from 15 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6816/22187,v1:172.21.15.42:6818/22187] [v2:172.21.15.42:6820/22187,v1:172.21.15.42:6822/22187] exists,up b6c53823-446f-4413-a2ff-d2ebe8fe9c71 2024-09-16T12:59:09.214 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 up in weight 1 up_from 15 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/22339,v1:172.21.15.136:6817/22339] [v2:172.21.15.136:6818/22339,v1:172.21.15.136:6819/22339] exists,up 278bfad1-010b-494e-ba61-bdf3935ca078 2024-09-16T12:59:09.214 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/22340,v1:172.21.15.136:6809/22340] [v2:172.21.15.136:6810/22340,v1:172.21.15.136:6811/22340] exists,up ed1af73b-b108-4189-8223-f017e634e3e8 2024-09-16T12:59:09.214 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/22364,v1:172.21.15.136:6825/22364] [v2:172.21.15.136:6826/22364,v1:172.21.15.136:6827/22364] exists,up ce904ba8-02e9-4579-9635-b4b6c6de1a08 2024-09-16T12:59:09.214 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/22341,v1:172.21.15.136:6801/22341] [v2:172.21.15.136:6802/22341,v1:172.21.15.136:6803/22341] exists,up d911da1b-2725-46a8-bbfa-8968de47271f 2024-09-16T12:59:09.226 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6817/22186,v1:172.21.15.42:6819/22186] [v2:172.21.15.42:6821/22186,v1:172.21.15.42:6823/22186] exists,up 0d8a8277-07cd-40f1-8c98-98b8b51d1689', 'osd.1 up in weight 1 up_from 14 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6808/22184,v1:172.21.15.42:6809/22184] [v2:172.21.15.42:6810/22184,v1:172.21.15.42:6811/22184] exists,up 6ce5996a-5e0e-4c9d-b268-27b34a14d830', 'osd.2 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6800/22185,v1:172.21.15.42:6801/22185] [v2:172.21.15.42:6802/22185,v1:172.21.15.42:6803/22185] exists,up 156042ab-909e-4368-b510-84018cb4354f', 'osd.3 up in weight 1 up_from 15 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6816/22187,v1:172.21.15.42:6818/22187] [v2:172.21.15.42:6820/22187,v1:172.21.15.42:6822/22187] exists,up b6c53823-446f-4413-a2ff-d2ebe8fe9c71', 'osd.4 up in weight 1 up_from 15 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/22339,v1:172.21.15.136:6817/22339] [v2:172.21.15.136:6818/22339,v1:172.21.15.136:6819/22339] exists,up 278bfad1-010b-494e-ba61-bdf3935ca078', 'osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/22340,v1:172.21.15.136:6809/22340] [v2:172.21.15.136:6810/22340,v1:172.21.15.136:6811/22340] exists,up ed1af73b-b108-4189-8223-f017e634e3e8', 'osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/22364,v1:172.21.15.136:6825/22364] [v2:172.21.15.136:6826/22364,v1:172.21.15.136:6827/22364] exists,up ce904ba8-02e9-4579-9635-b4b6c6de1a08', 'osd.7 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/22341,v1:172.21.15.136:6801/22341] [v2:172.21.15.136:6802/22341,v1:172.21.15.136:6803/22341] exists,up d911da1b-2725-46a8-bbfa-8968de47271f'] 2024-09-16T12:59:09.226 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-16T12:59:09.320 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:09.320 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mon.a", 2024-09-16T12:59:09.320 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster": "ceph", 2024-09-16T12:59:09.320 INFO:teuthology.orchestra.run.smithi042.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-16T12:59:09.320 INFO:teuthology.orchestra.run.smithi042.stdout: "admin_socket_mode": "", 2024-09-16T12:59:09.320 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-16T12:59:09.320 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_client_required": "cephx, none", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_cluster_required": "cephx", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_debug": "true", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_service_required": "cephx", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_supported": "cephx", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio": "true", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_poll_ms": "250", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_reap_max": "16", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_async_discard": "false", 2024-09-16T12:59:09.321 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_block_size": "4096", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio": "false", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_inflight_ios": "false", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_enable_discard": "false", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_flock_retry": "3", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_inject_crash": "0", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring": "false", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring_hipri": "false", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-16T12:59:09.322 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_type": "", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_alloc_size": "1048576", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_allocator": "hybrid", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_buffered_io": "true", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_for_zeros": "false", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_compact_log_sync": "false", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-16T12:59:09.323 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_max_log_runway": "4194304", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_max_prefetch": "1048576", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_min_flush_size": "524288", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_min_log_runway": "1048576", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_replay_recovery": "false", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_sync_write": "false", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-16T12:59:09.324 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_allocation_from_file": "true", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_allocator": "hybrid", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_async_db_compaction": "true", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_create": "true", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_create": "false", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_path": "", 2024-09-16T12:59:09.325 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_size": "0", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_path": "", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_preallocate_file": "false", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_size": "107374182400", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_create": "false", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_path": "", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_size": "100663296", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs": "true", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-16T12:59:09.326 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_autotune": "true", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size": "0", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-16T12:59:09.327 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-16T12:59:09.328 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_type": "2q", 2024-09-16T12:59:09.328 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-16T12:59:09.328 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_clone_cow": "true", 2024-09-16T12:59:09.328 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-16T12:59:09.328 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-16T12:59:09.328 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-16T12:59:09.328 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-16T12:59:09.328 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-16T12:59:09.328 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-16T12:59:09.328 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_mode": "none", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_csum_type": "crc32c", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_freelist": "false", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-16T12:59:09.329 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_small_allocations": "0", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_default_buffered_read": "true", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_default_buffered_write": "false", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-16T12:59:09.330 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fail_eio": "false", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-16T12:59:09.331 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-16T12:59:09.332 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mount": "false", 2024-09-16T12:59:09.332 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-16T12:59:09.332 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_umount": "false", 2024-09-16T12:59:09.332 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-16T12:59:09.332 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-16T12:59:09.332 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-16T12:59:09.332 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-16T12:59:09.332 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_ignore_data_csum": "false", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_op_age": "5.000000", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_alloc_size": "0", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size": "0", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-16T12:59:09.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_deferred_txc": "32", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size": "0", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_nid_prealloc": "1024", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_retry_disk_reads": "3", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_cf": "true", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-16T12:59:09.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_mem": "512", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_tracing": "false", 2024-09-16T12:59:09.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_zero_block_detection": "false", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-16T12:59:09.336 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_cluster_require_signatures": "false", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_cluster_require_version": "2", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_require_signatures": "false", 2024-09-16T12:59:09.337 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_require_version": "2", 2024-09-16T12:59:09.338 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_service_require_signatures": "false", 2024-09-16T12:59:09.338 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_service_require_version": "2", 2024-09-16T12:59:09.338 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_sign_messages": "true", 2024-09-16T12:59:09.338 INFO:teuthology.orchestra.run.smithi042.stdout: "chdir": "", 2024-09-16T12:59:09.338 INFO:teuthology.orchestra.run.smithi042.stdout: "client_acl_type": "", 2024-09-16T12:59:09.338 INFO:teuthology.orchestra.run.smithi042.stdout: "client_asio_thread_count": "2", 2024-09-16T12:59:09.338 INFO:teuthology.orchestra.run.smithi042.stdout: "client_cache_mid": "0.750000", 2024-09-16T12:59:09.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_cache_size": "16384", 2024-09-16T12:59:09.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_caps_release_delay": "5", 2024-09-16T12:59:09.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_check_pool_perm": "true", 2024-09-16T12:59:09.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-16T12:59:09.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_force_sync_read": "false", 2024-09-16T12:59:09.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_getattr_caps": "false", 2024-09-16T12:59:09.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_inject_features": "", 2024-09-16T12:59:09.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_inject_tick_delay": "0", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_die_on_failed_remount": "false", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_dirsize_rbytes": "true", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_force_lazyio": "false", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_fs": "", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_inject_release_failure": "false", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_max_inline_size": "4096", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mds_namespace": "", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_metadata": "", 2024-09-16T12:59:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_gid": "-1", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_timeout": "300", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_uid": "-1", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mountpoint": "/", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_notify_timeout": "10", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc": "true", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_dirty": "104857600", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_objects": "1000", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_size": "209715200", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_target_dirty": "8388608", 2024-09-16T12:59:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_permissions": "true", 2024-09-16T12:59:09.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_quota": "true", 2024-09-16T12:59:09.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_quota_df": "true", 2024-09-16T12:59:09.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_max_bytes": "0", 2024-09-16T12:59:09.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_max_periods": "4", 2024-09-16T12:59:09.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_min": "131072", 2024-09-16T12:59:09.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_reconnect_stale": "false", 2024-09-16T12:59:09.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_shutdown_timeout": "30", 2024-09-16T12:59:09.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_snapdir": ".snap", 2024-09-16T12:59:09.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_tick_interval": "1", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_trace": "", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_try_dentry_invalidate": "false", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_use_faked_inos": "false", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_use_random_mds": "false", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog": "false", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog_port": "12201", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_monitors": "default=true", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog": "false", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-16T12:59:09.343 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog_level": "info", 2024-09-16T12:59:09.344 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "-", 2024-09-16T12:59:09.344 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_network": "", 2024-09-16T12:59:09.344 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_network_interface": "", 2024-09-16T12:59:09.344 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_isal": "false", 2024-09-16T12:59:09.344 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_level": "5", 2024-09-16T12:59:09.344 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_winsize": "-15", 2024-09-16T12:59:09.344 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zstd_level": "1", 2024-09-16T12:59:09.344 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_alien_op_num_threads": "6", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location": "", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location_hook": "", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location_hook_timeout": "10", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "daemonize": "false", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_alienstore": "0/5", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_allow_any_pool_priority": "false", 2024-09-16T12:59:09.345 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asok": "1/5", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asok_assert_abort": "false", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asserts_on_shutdown": "true", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_auth": "20/20", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bdev": "1/3", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bluefs": "1/5", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bluestore": "1/5", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_buffer": "0/1", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_ceph_exporter": "1/5", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_cephfs_mirror": "0/5", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_cephsqlite": "0/5", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_civetweb": "1/10", 2024-09-16T12:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_client": "0/5", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_compressor": "1/5", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_context": "0/1", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_crush": "1/1", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_crypto": "1/5", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_deliberately_leak_memory": "false", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_disable_randomized_ping": "false", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_dpdk": "1/5", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_eventtrace": "1/5", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_filer": "0/1", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_filestore": "1/3", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_finisher": "1/1", 2024-09-16T12:59:09.347 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_fuse": "1/5", 2024-09-16T12:59:09.348 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_heartbeat_testing_span": "0", 2024-09-16T12:59:09.348 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_heartbeatmap": "1/5", 2024-09-16T12:59:09.348 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-16T12:59:09.348 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_javaclient": "1/5", 2024-09-16T12:59:09.348 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_journal": "1/3", 2024-09-16T12:59:09.348 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_journaler": "0/5", 2024-09-16T12:59:09.348 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_kstore": "1/5", 2024-09-16T12:59:09.348 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_leveldb": "4/5", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_lockdep": "0/1", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mclock": "1/5", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds": "1/5", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_balancer": "1/5", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_locker": "1/5", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_log": "1/5", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_log_expire": "1/5", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_migrator": "1/5", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_memdb": "4/5", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mgr": "2/5", 2024-09-16T12:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mgrc": "1/5", 2024-09-16T12:59:09.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mon": "20/20", 2024-09-16T12:59:09.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_monc": "0/10", 2024-09-16T12:59:09.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_ms": "1/1", 2024-09-16T12:59:09.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_none": "0/5", 2024-09-16T12:59:09.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objclass": "0/5", 2024-09-16T12:59:09.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objectcacher": "0/5", 2024-09-16T12:59:09.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objecter": "0/1", 2024-09-16T12:59:09.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_optracker": "0/5", 2024-09-16T12:59:09.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_osd": "1/5", 2024-09-16T12:59:09.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_paxos": "20/20", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_perfcounter": "1/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_prioritycache": "1/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rados": "0/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd": "0/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_mirror": "0/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_pwl": "0/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_replay": "0/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_refs": "0/0", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_reserver": "1/1", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw": "1/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_datacache": "1/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_sync": "1/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rocksdb": "4/5", 2024-09-16T12:59:09.351 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_cache": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_cleaner": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_device": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_journal": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_lba": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_odata": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_omap": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_onode": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_tm": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_striper": "0/1", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_test": "0/5", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_throttle": "1/1", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_timer": "0/1", 2024-09-16T12:59:09.352 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_tp": "0/5", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "device_failure_prediction_mode": "none", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_graylog": "false", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_journald": "false", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_stderr": "true", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_syslog": "false", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event_tracing": "false", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_addr": "0.0.0.0", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_http_port": "9926", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_prio_limit": "5", 2024-09-16T12:59:09.353 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_sort_metrics": "true", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_stats_period": "5", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "fake_statfs_for_testing": "0", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "fatal_signal_handlers": "true", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "filer_max_purge_ops": "10", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "filer_max_truncate_ops": "128", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_apply_finisher_threads": "1", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_blackhole": "false", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_btrfs_clone_range": "true", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_btrfs_snap": "true", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_caller_concurrency": "10", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_collect_device_partition_information": "true", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_commit_timeout": "600.000000", 2024-09-16T12:59:09.354 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_inject_read_err": "false", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_omap_check": "false", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_verify_split": "false", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_dump_file": "", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fadvise": "true", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fail_eio": "true", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fd_cache_shards": "16", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fd_cache_size": "128", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fiemap": "false", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fiemap_threshold": "4096", 2024-09-16T12:59:09.355 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_inject_stall": "0", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_parallel": "false", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_trailing": "false", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_writeahead": "false", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_kill_at": "0", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs": "0", 2024-09-16T12:59:09.356 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size": "0", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_merge_threshold": "-10", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_odsync_write": "false", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_backend": "rocksdb", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_backend_path": "", 2024-09-16T12:59:09.357 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_thread_timeout": "60", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_threads": "2", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_punch_hole": "false", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-16T12:59:09.358 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_ops": "50", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_seek_data_hole": "false", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_sloppy_crc": "false", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_splice": "false", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_split_multiple": "2", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_split_rand_factor": "20", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_update_to": "1000", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-16T12:59:09.359 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_enable": "true", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_xfs_extsize": "false", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_zfs_snap": "false", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "fio_dir": "/tmp/fio", 2024-09-16T12:59:09.360 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7", 2024-09-16T12:59:09.361 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_allow_other": "true", 2024-09-16T12:59:09.361 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_atomic_o_trunc": "true", 2024-09-16T12:59:09.361 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_big_writes": "true", 2024-09-16T12:59:09.361 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_debug": "false", 2024-09-16T12:59:09.361 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_default_permissions": "false", 2024-09-16T12:59:09.361 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_disable_pagecache": "false", 2024-09-16T12:59:09.361 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_max_write": "0", 2024-09-16T12:59:09.361 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_multithreaded": "true", 2024-09-16T12:59:09.361 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_require_active_mds": "true", 2024-09-16T12:59:09.361 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_set_user_groups": "true", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_move": "true", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_read": "true", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_write": "true", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_use_invalidate_cb": "true", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "gss_target_name": "ceph", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_file": "", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_inject_failure": "0", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_interval": "5", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "host": "smithi042", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-16T12:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-16T12:59:09.363 INFO:teuthology.orchestra.run.smithi042.stdout: "inject_early_sigterm": "false", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "jaeger_tracing_enable": "false", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_aio": "true", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_align_min_size": "65536", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_block_align": "true", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_block_size": "4096", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_dio": "true", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_discard": "false", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_force_aio": "false", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_ignore_corruption": "false", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_max_write_bytes": "10485760", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_max_write_entries": "100", 2024-09-16T12:59:09.364 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_replay_from": "0", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_write_header_frequency": "0", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_zero_on_create": "false", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_prefetch_periods": "10", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_prezero_periods": "5", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_write_head_interval": "15", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "keyfile": "", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_backend": "rocksdb", 2024-09-16T12:59:09.365 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_default_stripe_size": "65536", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_fsck_on_mount": "false", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_max_bytes": "67108864", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_max_ops": "512", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_nid_prealloc": "1024", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_onode_map_size": "1024", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_sync_submit_transaction": "false", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_sync_transaction": "false", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_block_size": "65536", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_bloom_size": "0", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_cache_size": "536870912", 2024-09-16T12:59:09.366 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_compact_on_mount": "false", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_compression": "false", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_log": "", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_max_open_files": "0", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_paranoid": "false", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "librados_thread_count": "2", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "lockdep": "false", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "lockdep_force_backtrace": "false", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "log_coarse_timestamps": "true", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "log_flush_on_exit": "false", 2024-09-16T12:59:09.367 INFO:teuthology.orchestra.run.smithi042.stdout: "log_graylog_host": "127.0.0.1", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_graylog_port": "12201", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_max_new": "1000", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_max_recent": "10000", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_stderr_prefix": "", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_stop_at_utilization": "0.970000", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": "true", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_graylog": "false", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_journald": "false", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_stderr": "false", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_syslog": "false", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "max_rotating_auth_attempts": "10", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-16T12:59:09.368 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_action_on_write_error": "1", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_alternate_name_max": "8192", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_asio_thread_count": "2", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_export_pin": "true", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_dirs": "true", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_interval": "5", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_interval": "10", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_max": "-1", 2024-09-16T12:59:09.369 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_max_until": "-1", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_merge_size": "50", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_midchunk": "0.300000", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_min_start": "0.200000", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_minchunk": "0.001000", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_mode": "0", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_need_max": "1.200000", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_need_min": "0.800000", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_bits": "3", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_size": "10000", 2024-09-16T12:59:09.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_target_decay": "10.000000", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_grace": "15.000000", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_interval": "4.000000", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_mid": "0.700000", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_release_free_interval": "10", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_reservation": "0.050000", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_interval": "1", 2024-09-16T12:59:09.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_threshold": "262144", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_prealloc_inos": "1000", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_connect_bootstrapping": "false", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_damage_table_max_entries": "10000", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_auth_pins": "false", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_frag": "false", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_scatterstat": "false", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_subtrees": "false", 2024-09-16T12:59:09.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_decay_halflife": "5.000000", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_default_dir_hash": "2", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_defer_session_stale": "true", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_deny_all_reconnect": "false", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_keys_per_op": "16384", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_max_commit_size": "10", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_max_entries": "0", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_on_map": "false", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_early_reply": "true", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_enable_op_tracker": "true", 2024-09-16T12:59:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_enforce_unique_name": "true", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_random": "true", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_health_summarize_threshold": "10", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_health_dummy": "false", 2024-09-16T12:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_migrator_session_race": "false", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_join_fs": "", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_journal_format": "1", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_create_at": "0", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_export_at": "0", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_import_at": "0", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_at": "0", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_expire_at": "0", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_replay_at": "0", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_link_at": "0", 2024-09-16T12:59:09.375 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_mdstable_at": "0", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_openc_at": "0", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_rename_at": "0", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_events_per_segment": "1024", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_max_events": "-1", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_max_segments": "128", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_pause": "false", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_segment_size": "0", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_warn_factor": "2.000000", 2024-09-16T12:59:09.376 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_caps_per_client": "1048576", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_completed_flushes": "100000", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_completed_requests": "100000", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_export_size": "20971520", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_file_recover": "32", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_files": "64", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_ops": "8192", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_snaps_per_dir": "100", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_metrics_update_interval": "2", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_min_caps_per_client": "100", 2024-09-16T12:59:09.377 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_min_caps_working_set": "10000", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_numa_node": "-1", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_complaint_time": "30.000000", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_duration": "600", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_size": "20", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_slow_op_size": "20", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_log_threshold": "5", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_ping_grace": "15", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_ping_interval": "5", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-16T12:59:09.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_caps": "30000", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_warning_threshold": "262144", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_replay_interval": "1.000000", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_root_ino_gid": "0", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_root_ino_uid": "0", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-16T12:59:09.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_shutdown_check": "0", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_skip_ino": "0", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_max_uid": "4294967294", 2024-09-16T12:59:09.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_min_uid": "0", 2024-09-16T12:59:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_rstat": "false", 2024-09-16T12:59:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_standby_replay_damaged": "false", 2024-09-16T12:59:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_symlink_recovery": "true", 2024-09-16T12:59:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-16T12:59:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_thrash_exports": "0", 2024-09-16T12:59:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_thrash_fragments": "0", 2024-09-16T12:59:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_tick_interval": "5.000000", 2024-09-16T12:59:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_valgrind_exit": "false", 2024-09-16T12:59:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_verify_backtrace": "1", 2024-09-16T12:59:09.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_verify_scatter": "false", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_wipe_sessions": "false", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool_debug": "false", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_device_bytes": "1073741824", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_page_set": "false", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_page_size": "65536", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_bytes": "134217728", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_messages": "512", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-16T12:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_max_pg_num_change": "128", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mds_bytes": "134217728", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mds_messages": "128", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mon_bytes": "134217728", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mon_messages": "128", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_osd_bytes": "536870912", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_osd_messages": "8192", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_pool": "true", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_standby_modules": "true", 2024-09-16T12:59:09.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_stats_period": "5", 2024-09-16T12:59:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_stats_threshold": "5", 2024-09-16T12:59:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_tick_period": "2", 2024-09-16T12:59:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-16T12:59:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-16T12:59:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_allow_pool_delete": "true", 2024-09-16T12:59:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_allow_pool_size_one": "true", 2024-09-16T12:59:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_auth_validate_all_caps": "true", 2024-09-16T12:59:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-16T12:59:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-16T12:59:09.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_bytes": "104857600", 2024-09-16T12:59:09.385 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_directed_command_retry": "5", 2024-09-16T12:59:09.385 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-16T12:59:09.385 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-16T12:59:09.385 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-16T12:59:09.385 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-16T12:59:09.385 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_parallel": "3", 2024-09-16T12:59:09.385 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_log_interval": "1.000000", 2024-09-16T12:59:09.385 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-16T12:59:09.385 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_ping_interval": "10.000000", 2024-09-16T12:59:09.385 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_level": "debug", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_file": "true", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_journald": "false", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_bootstrap": "false", 2024-09-16T12:59:09.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_start": "false", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_trim": "true", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cpu_threads": "4", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_crush_min_required_version": "hammer", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_daemon_bytes": "419430400", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_avail_crit": "5", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_avail_warn": "5", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_size_warn": "16106127360", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-16T12:59:09.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_json": "false", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_transactions": "false", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_extra_checks": "false", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_pacific": "false", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_quincy": "false", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_down_mkfs_grace": "120", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_election_default_strategy": "1", 2024-09-16T12:59:09.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_election_timeout": "5.000000", 2024-09-16T12:59:09.389 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-16T12:59:09.389 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ping_divisor": "2", 2024-09-16T12:59:09.389 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-16T12:59:09.389 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_enable_op_tracker": "true", 2024-09-16T12:59:09.389 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_fake_pool_delete": "false", 2024-09-16T12:59:09.389 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_force_quorum_join": "false", 2024-09-16T12:59:09.389 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_globalid_prealloc": "10000", 2024-09-16T12:59:09.389 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_detail_to_clog": "false", 2024-09-16T12:59:09.389 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_log_update_period": "5", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_max_detail": "50", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog": "true", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog_interval": "600", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_host": "172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.136", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_host_override": "", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_initial_members": "", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-16T12:59:09.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease": "5.000000", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_full_interval": "50", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_max": "10000", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_max_summary": "50", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_log_epochs": "500", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_osd": "10000", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_pg_per_osd": "10000", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_pool_pg_num": "65536", 2024-09-16T12:59:09.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_force_trim_to": "0", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_skip_sanity": "false", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_memory_autotune": "true", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_memory_target": "2147483648", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_beacon_grace": "30", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_digest_period": "5", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_inactive_grace": "60", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-16T12:59:09.392 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_min_osdmap_epochs": "500", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_complaint_time": "30", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_duration": "600", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_size": "20", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_slow_op_size": "20", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_log_threshold": "5", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_in": "false", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-16T12:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-16T12:59:09.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-16T12:59:09.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_cache_size": "500", 2024-09-16T12:59:09.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-16T12:59:09.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-16T12:59:09.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-16T12:59:09.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_down_out_interval": "600", 2024-09-16T12:59:09.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-16T12:59:09.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-16T12:59:09.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_force_trim_to": "0", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_down_reporters": "2", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-16T12:59:09.395 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_report_timeout": "900", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-16T12:59:09.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_stuck_threshold": "60", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_probe_timeout": "2.000000", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_max_change": "0.050000", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_max_osds": "4", 2024-09-16T12:59:09.397 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-16T12:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-16T12:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-16T12:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-16T12:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-16T12:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_interval": "86400", 2024-09-16T12:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_max_keys": "100", 2024-09-16T12:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_timeout": "300", 2024-09-16T12:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_session_timeout": "300", 2024-09-16T12:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_smart_report_timeout": "5", 2024-09-16T12:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stat_smooth_intervals": "6", 2024-09-16T12:59:09.399 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-16T12:59:09.399 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_pool_min_size": "2", 2024-09-16T12:59:09.399 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_pool_size": "4", 2024-09-16T12:59:09.399 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-16T12:59:09.399 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-16T12:59:09.399 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_debug": "false", 2024-09-16T12:59:09.399 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-16T12:59:09.399 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-16T12:59:09.400 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_provider_kill_at": "0", 2024-09-16T12:59:09.400 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_requester_kill_at": "0", 2024-09-16T12:59:09.400 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_timeout": "60.000000", 2024-09-16T12:59:09.400 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_target_pg_per_osd": "100", 2024-09-16T12:59:09.400 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_tick_interval": "5", 2024-09-16T12:59:09.400 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_timecheck_interval": "300.000000", 2024-09-16T12:59:09.400 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-16T12:59:09.400 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_older_version_delay": "604800", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_misplaced": "false", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_older_version": "true", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-16T12:59:09.401 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": "", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_op_threads": "3", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_cm": "false", 2024-09-16T12:59:09.402 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_device_name": "", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_dscp": "96", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_local_gid": "", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_port_num": "1", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_sl": "3", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_support_srq": "true", 2024-09-16T12:59:09.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_type": "ib", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_reap_threshold": "5", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_before_connect": "false", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_ipv4": "true", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_ipv6": "false", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_msgr1": "true", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_msgr2": "true", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_port_max": "7568", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_port_min": "6800", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_retry_count": "3", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_retry_delay": "5", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_client": "false", 2024-09-16T12:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mds": "false", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mgr": "false", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mon": "false", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_osd": "false", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_client_mode": "crc secure", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_cluster_mode": "crc secure", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_cluster_type": "", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_compress_secure": "false", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_connection_idle_timeout": "900", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_connection_ready_timeout": "10", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_crc_data": "true", 2024-09-16T12:59:09.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_crc_header": "true", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_bad_msg": "false", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_bug": "true", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_old_message": "true", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_skipped_message": "false", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_coremask": "0xF", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_enable_tso": "true", 2024-09-16T12:59:09.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hugepages": "", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_lro": "true", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_memory_channel": "4", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_pmd": "", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_port_id": "0", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-16T12:59:09.407 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dump_on_send": "false", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_initial_backoff": "0.200000", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_max": "1.000000", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_type": "", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_socket_failures": "5000", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_learn_addr_from_peer": "true", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_max_accept_failures": "4", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_max_backoff": "15.000000", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_client_mode": "secure crc", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-16T12:59:09.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_service_mode": "secure crc", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compress_min_size": "1024", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compress_mode": "none", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_pq_min_cost": "65536", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_public_type": "", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_service_mode": "crc secure", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_listen_backlog": "512", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_nodelay": "true", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_rcvbuf": "0", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_type": "async+posix", 2024-09-16T12:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout: "no_config_file": "false", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_completion_locks_per_session": "32", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inflight_ops": "1024", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_tick_interval": "5.000000", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_timeout": "10.000000", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "objectstore_blackhole": "false", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "openssl_engine_opts": "", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_delay_time": "5.000000", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_hist_halflife": "1000", 2024-09-16T12:59:09.410 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_max_low_ops": "2", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_max_ops": "4", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_slop": "0.020000", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_asio_thread_count": "2", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_async_recovery_min_cost": "100", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_scan_max": "512", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_scan_min": "64", 2024-09-16T12:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_degraded": "false", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_peering": "false", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_unfound": "true", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_beacon_report_interval": "300", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_duration": "30", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_max_block_size": "67108864", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_blkin_trace_all": "false", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_check_for_log_corruption": "false", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-16T12:59:09.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_update_on_start": "true", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_message_cap": "256", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_message_size_cap": "524288000", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_op_priority": "63", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_watch_timeout": "30", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_command_max_records": "256", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_compact_on_start": "false", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-16T12:59:09.413 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_update_on_start": "true", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_update_weight_set": "true", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_feed_pullee": "-1", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_misdirected_ops": "false", 2024-09-16T12:59:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_no_acting_change": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_no_purge_strays": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_op_order": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_shutdown": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_snaps": "false", 2024-09-16T12:59:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_keys": "1024", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_stride": "524288", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_default_notify_timeout": "30", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep": "0.000000", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-16T12:59:09.416 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_discard_disconnected_ops": "true", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_enable_op_tracker": "true", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_info": "true", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown": "true", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-16T12:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_function_tracing": "false", 2024-09-16T12:59:09.418 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_grace": "20", 2024-09-16T12:59:09.418 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_interval": "6", 2024-09-16T12:59:09.418 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-16T12:59:09.418 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_peers": "10", 2024-09-16T12:59:09.418 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_size": "2000", 2024-09-16T12:59:09.418 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_stale": "600", 2024-09-16T12:59:09.418 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_max_size": "100000", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_min_size": "1000", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal_size": "5120", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_kill_backfill_at": "0", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_cache_size": "50", 2024-09-16T12:59:09.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_dedup": "true", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_message_max": "40", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_share_max_epochs": "40", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_attr_name_len": "100", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_attr_size": "0", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_backfills": "1", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_markdown_count": "5", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_markdown_period": "600", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_name_len": "2048", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_namespace_len": "256", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_size": "134217728", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-16T12:59:09.420 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_blocked_by": "16", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_log_entries": "10000", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pgls": "1024", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_push_cost": "8388608", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_push_objects": "10", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_scrubs": "3", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_trimming_pgs": "2", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_write_op_reply_len": "64", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_write_size": "90", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-16T12:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_profile": "balanced", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-16T12:59:09.422 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_base": "805306368", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_cache_min": "134217728", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target": "4294967296", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target_autotune": "false", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-16T12:59:09.423 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_min_pg_log_entries": "250", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_min_recovery_priority": "0", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_report_interval": "5", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_num_cache_shards": "32", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_num_op_tracker_shard": "32", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_auto_affinity": "true", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_node": "-1", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_prefer_iface": "true", 2024-09-16T12:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objecter_finishers": "1", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore": "bluestore", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore_fuse": "false", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore_tracing": "false", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_complaint_time": "30.000000", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_duration": "600", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_size": "20", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_slow_op_size": "20", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_log_threshold": "5", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards": "0", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards_hdd": "5", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards_ssd": "8", 2024-09-16T12:59:09.425 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_pq_min_cost": "65536", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_queue_cut_off": "high", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_thread_timeout": "15", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_open_classes_on_start": "true", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_os_flags": "0", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_peering_op_priority": "255", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_delete_cost": "1048576", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_delete_priority": "5", 2024-09-16T12:59:09.426 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_trim_max": "10000", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_trim_min": "100", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-16T12:59:09.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flags": "0", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_min_size": "0", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pg_num": "32", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pgp_num": "0", 2024-09-16T12:59:09.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-16T12:59:09.429 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_size": "2", 2024-09-16T12:59:09.429 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_type": "replicated", 2024-09-16T12:59:09.429 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-16T12:59:09.429 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-16T12:59:09.429 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_push_per_object_cost": "1000", 2024-09-16T12:59:09.429 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-16T12:59:09.429 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recover_clone_overlap": "true", 2024-09-16T12:59:09.429 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-16T12:59:09.429 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_cost": "20971520", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active": "0", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_single_start": "1", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_op_priority": "3", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_priority": "5", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep": "0.000000", 2024-09-16T12:59:09.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_repair_during_recovery": "false", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_requested_scrub_priority": "120", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_auto_repair": "false", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_begin_hour": "0", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_begin_week_day": "0", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_chunk_max": "25", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_chunk_min": "5", 2024-09-16T12:59:09.431 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_cost": "52428800", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_during_recovery": "false", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_end_hour": "0", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_end_week_day": "0", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_event_cost": "4096", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_invalid_stats": "true", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_max_preemptions": "5", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-16T12:59:09.432 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_priority": "5", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_sleep": "0.000000", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_skip_data_digest": "false", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_smart_report_timeout": "5", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_cost": "1048576", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_priority": "5", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-16T12:59:09.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_target_transaction_size": "30", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tracing": "false", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_use_stale_snap": "false", 2024-09-16T12:59:09.434 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "osdc_blkin_trace_all": "false", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_kill_at": "0", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_max_join_drift": "10", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_min": "500", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_min_wait": "0.050000", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_propose_interval": "1.000000", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_max": "500", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_min": "250", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_stash_full_interval": "25", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_trim_max": "500", 2024-09-16T12:59:09.435 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_trim_min": "250", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "perf": "true", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "-", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrv": "", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "public_bind_addr": "-", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "public_network": "", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "public_network_interface": "", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "qat_compressor_enabled": "false", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_mon_op_timeout": "0", 2024-09-16T12:59:09.436 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_osd_op_timeout": "0", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_tracing": "false", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_atime_update_interval": "60", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_balance_parent_reads": "false", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_balance_snap_reads": "false", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blkin_trace_all": "false", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache": "true", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-16T12:59:09.437 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_policy": "writearound", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_size": "33554432", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_clone_copy_on_read": "false", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_compression_hint": "none", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_concurrent_management_ops": "10", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_clone_format": "auto", 2024-09-16T12:59:09.438 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_data_pool": "", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_features": "61", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_format": "2", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_map_options": "", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_order": "22", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_pool": "rbd", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_stripe_count": "0", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_stripe_unit": "0", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_enable_alloc_hint": "true", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-16T12:59:09.439 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_io_scheduler": "simple", 2024-09-16T12:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-16T12:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-16T12:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-16T12:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-16T12:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-16T12:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-16T12:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-16T12:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-16T12:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-16T12:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_order": "24", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_pool": "", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_splay_width": "4", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_localize_parent_reads": "false", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_localize_snap_reads": "false", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-16T12:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-16T12:59:09.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-16T12:59:09.443 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mtime_update_interval": "60", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_non_blocking_aio": "true", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_op_thread_timeout": "60", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_op_threads": "1", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_parent_cache_enabled": "false", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_plugins": "", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_burst": "0", 2024-09-16T12:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_limit": "0", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_exclude_ops": "0", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_burst": "0", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_limit": "0", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-16T12:59:09.445 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_read_from_replica_policy": "default", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_skip_partial_discard": "true", 2024-09-16T12:59:09.446 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_tracing": "false", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_validate_names": "true", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_validate_pool": "true", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "restapi_base_url": "", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "restapi_log_level": "", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_acl_grants_max_num": "100", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_admin_entry": "admin", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_backend_store": "rados", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_barbican_url": "", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_beast_enable_async": "true", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-16T12:59:09.447 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_enabled": "true", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_expiry_interval": "900", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_lru_size": "10000", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_content_length_compat": "false", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_copy_obj_progress": "true", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cors_rules_max_num": "100", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cross_domain_policy": "", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-16T12:59:09.448 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_addr": "", 2024-09-16T12:59:09.449 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-16T12:59:09.449 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-16T12:59:09.449 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-16T12:59:09.449 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-16T12:59:09.449 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_password": "", 2024-09-16T12:59:09.449 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-16T12:59:09.449 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_username": "", 2024-09-16T12:59:09.449 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_require_ssl": "true", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-16T12:59:09.450 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_addr": "", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_auth": "token", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_namespace": "", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_prefix": "", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_token_file": "", 2024-09-16T12:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_buffersize": "524288", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_low_speed_time": "300", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-16T12:59:09.452 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_changes_size": "1000", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_num_shards": "128", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_window": "30", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-16T12:59:09.453 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dns_name": "", 2024-09-16T12:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dns_s3website_name": "", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dynamic_resharding": "true", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_gc_threads": "true", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_lc_threads": "true", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_ops_log": "false", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_quota_threads": "true", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_static_website": "false", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_usage_log": "false", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enforce_swift_acls": "true", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_exit_timeout_secs": "120", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_expose_bucket": "false", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_extended_http_attrs": "", 2024-09-16T12:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_frontends": "beast port=7480", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_deferred": "50", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_objs": "32", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_processor_period": "3600", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-16T12:59:09.456 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_init_timeout": "300", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_kafka_connection_idle": "300", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_domain": "", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_password": "", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_password_path": "", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_project": "", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_tenant": "", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_token": "", 2024-09-16T12:59:09.457 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_token_path": "", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_user": "", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_api_version": "2", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_domain": "", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_password": "", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_project": "", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_user": "", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-16T12:59:09.458 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_url": "", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_debug_interval": "-1", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_lock_max_time": "90", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_objs": "32", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_rules": "1000", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_worker": "3", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_thread_delay": "0", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_dnattr": "uid", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_searchfilter": "", 2024-09-16T12:59:09.459 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_uri": "", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_http_headers": "", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_object_name_utc": "false", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attr_name_len": "0", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attr_size": "0", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_chunk_size": "4194304", 2024-09-16T12:59:09.460 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_listing_results": "1000", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_notify_retries": "10", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_put_param_size": "1048576", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_put_size": "5368709120", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_slo_entries": "1000", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_md_log_max_shards": "64", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-16T12:59:09.461 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_mp_lock_max_time": "600", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_max_gc": "300", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-16T12:59:09.462 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_num_async_rados_threads": "32", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_num_control_oids": "8", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_numa_node": "-1", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_chunk_size": "100", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_gc_interval": "600", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_thread_timeout": "600", 2024-09-16T12:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_tracing": "false", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_token": "", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_url": "", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_verify_ssl": "true", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_file_path": "", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_rados": "true", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_socket_path": "", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_push_interval": "2.000000", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-16T12:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_print_continue": "true", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_tracing": "false", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm": "", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm_id": "", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_region": "", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-16T12:59:09.465 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_request_uri": "", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_batch_size": "64", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_max_aio": "128", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_num_logs": "16", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_thread_interval": "600", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_resolve_cname": "false", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_run_sync_thread": "true", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-16T12:59:09.466 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_scheduler_type": "throttler", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_script_uri": "", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_service_provider_name": "", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_client_id": "", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_client_secret": "", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_entry": "sts", 2024-09-16T12:59:09.467 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_key": "sts", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_min_session_duration": "900", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_token_introspection_url": "", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_account_in_url": "false", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_auth_entry": "auth", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_auth_url": "", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_custom_header": "", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_need_stats": "true", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_tenant_name": "", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_token_expiration": "86400", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_url": "", 2024-09-16T12:59:09.468 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_url_prefix": "swift", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_lease_period": "120", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-16T12:59:09.469 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_thread_pool_size": "512", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_comment": "", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_createby": "", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_encoding": "", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_flag": "false", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_origin": "", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_tracker": "", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_trust_forwarded_https": "false", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_max_shards": "32", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_max_user_shards": "1", 2024-09-16T12:59:09.470 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_use_opa_authz": "false", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_max_buckets": "1000", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_unique_email": "true", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_verify_ssl": "true", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone": "", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone_id": "", 2024-09-16T12:59:09.471 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup": "", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup_id": "", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_block_size": "4096", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_size": "536870912", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-16T12:59:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_index_type": "binary_search", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_partition_filters": "false", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_perf": "false", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-16T12:59:09.473 INFO:teuthology.orchestra.run.smithi042.stdout: "run_dir": "/var/run/ceph", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_block_create": "true", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_cache_lru_size": "67108864", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_default_max_object_size": "16777216", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_device_size": "107374182400", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_capacity": "16", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_segment_size": "67108864", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "setgroup": "", 2024-09-16T12:59:09.474 INFO:teuthology.orchestra.run.smithi042.stdout: "setuser": "", 2024-09-16T12:59:09.475 INFO:teuthology.orchestra.run.smithi042.stdout: "setuser_match_path": "", 2024-09-16T12:59:09.475 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-16T12:59:09.475 INFO:teuthology.orchestra.run.smithi042.stdout: "thp": "false", 2024-09-16T12:59:09.475 INFO:teuthology.orchestra.run.smithi042.stdout: "threadpool_default_timeout": "60", 2024-09-16T12:59:09.475 INFO:teuthology.orchestra.run.smithi042.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-16T12:59:09.475 INFO:teuthology.orchestra.run.smithi042.stdout: "throttler_perf_counter": "true" 2024-09-16T12:59:09.475 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:09.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-16T12:59:09.617 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a: {} 2024-09-16T12:59:09.635 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b: {} 2024-09-16T12:59:09.656 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c: {} 2024-09-16T12:59:09.659 INFO:teuthology.orchestra.run.smithi042.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T12:59:09.659 INFO:teuthology.orchestra.run.smithi042.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T12:59:09.659 INFO:teuthology.orchestra.run.smithi042.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T12:59:09.672 DEBUG:teuthology.orchestra.run.smithi136:> type ceph-objectstore-tool 2024-09-16T12:59:09.678 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T12:59:09.679 DEBUG:teuthology.orchestra.run.smithi042:> type ceph-objectstore-tool 2024-09-16T12:59:09.685 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T12:59:09.685 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-16T12:59:09.695 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-16T12:59:09.695 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-16T12:59:09.695 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T12:59:09.695 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T12:59:09.695 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-16T12:59:09.696 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-16T12:59:09.696 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:09.696 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-16T12:59:09.697 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:09.697 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-16T12:59:09.697 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T12:59:09.697 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-16T12:59:09.722 INFO:teuthology.orchestra.run.smithi042.stdout:mapped writes DISABLED 2024-09-16T12:59:09.722 INFO:teuthology.orchestra.run.smithi042.stdout:mapped reads DISABLED 2024-09-16T12:59:09.722 INFO:teuthology.orchestra.run.smithi042.stdout:Seed set to 3443 2024-09-16T12:59:09.723 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:09.719+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:09.798 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:09.791+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:09.813 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:09.813 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:09.813 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:09.823 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:09.869 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:09.898 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:09.898 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:09.891+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:09.918 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:09.925 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:09.925 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:09.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:09.935 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:09.957 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:09.979 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:09.999 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:09.995+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:10.000 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:10.022 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:10.024 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:10.024 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:10.024 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:10.024 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:10.025 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:10.047 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:10.047 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:10.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:10.100 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:10.095+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:10.201 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:10.195+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:10.201 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:10.202 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:10.210 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:10.302 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:10.295+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:10.313 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:10.313 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:10.322 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:10.403 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:10.399+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:10.427 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:10.427 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:10.436 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:10.504 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:10.499+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:10.538 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:10.538 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:10.547 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:10.605 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:10.599+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:10.648 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:10.648 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:10.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:10.706 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:10.699+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:10.762 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T12:59:10.768 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T12:59:10.772 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T12:59:10.806 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:10.803+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:10.813 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:10.813 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:10.823 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:10.907 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:10.903+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:10.929 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:10.929 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:10.938 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:11.008 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:11.003+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:11.078 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:11.078 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:11.087 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:11.108 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:11.103+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:11.187 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:11.187 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:11.197 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:11.242 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:11.239+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:11.302 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:11.302 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:11.310 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:11.303+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:11.312 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:11.409 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:11.409 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:11.411 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:11.403+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:11.418 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:11.511 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:11.507+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:11.517 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:11.517 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:11.526 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:11.612 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:11.607+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:11.672 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:11.672 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:11.682 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:11.713 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:11.707+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T12:59:11.783 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T12:59:11.785 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:11.785 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:11.797 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:11.827 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:11.823+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:11.896 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:11.896 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:11.905 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:11.914 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:11.907+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:12.001 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:12.001 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:12.010 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:12.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:12.054 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:12.051+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:12.116 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:12.111+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:12.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:12.159 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:12.166 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:12.168 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:12.185 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:12.208 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:12.216 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:12.211+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:12.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:12.251 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:12.265 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:12.265 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:12.272 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:12.275 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:12.290 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:12.313 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:12.316 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:12.318 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:12.311+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:12.372 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:12.372 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:12.372 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:12.372 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:12.381 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:12.419 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:12.411+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:12.477 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:12.477 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:12.477 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:12.477 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:12.477 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:12.487 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:12.519 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:12.515+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:12.593 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.037755560000001, 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T12:59:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T12:59:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.212928976000001, 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010809278, 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374549+0000", 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.037e-06 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374711+0000", 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016226599999999999 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374734+0000", 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2895999999999999e-05 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374746+0000", 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.221e-05 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.380942+0000", 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061959390000000001 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381063+0000", 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121274 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381255+0000", 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019110500000000001 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384487+0000", 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032327979999999998 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385296+0000", 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080888499999999996 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385332+0000", 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5774e-05 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385357+0000", 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5094000000000001e-05 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.212718083999999, 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010976025, 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 45 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374760+0000", 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.13e-06 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374769+0000", 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5699999999999999e-06 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374819+0000", 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9882000000000001e-05 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374828+0000", 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382329+0000", 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075009009999999999 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382403+0000", 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4208999999999994e-05 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382514+0000", 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111276 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385533+0000", 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030190080000000001 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385564+0000", 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0926000000000002e-05 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385666+0000", 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101791 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385735+0000", 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8541999999999995e-05 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.212638513000002, 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T12:59:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.212591257, 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T12:59:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T12:59:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.212388320999999, 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.212357058999999, 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T12:59:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T12:59:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.212321296999999, 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T12:59:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T12:59:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T12:59:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.212126667, 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T12:59:12.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T12:59:12.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.212079822, 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T12:59:12.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.668019781000002, 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010623719, 2024-09-16T12:59:12.656 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 69 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919458+0000", 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.474e-06 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919621+0000", 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016234800000000001 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919641+0000", 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0336999999999999e-05 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919656+0000", 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4554e-05 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924273+0000", 2024-09-16T12:59:12.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046169610000000002 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924411+0000", 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138286 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924627+0000", 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021597399999999999 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928794+0000", 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004167268 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929983+0000", 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011884409999999999 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930039+0000", 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6363000000000002e-05 2024-09-16T12:59:12.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930081+0000", 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1712999999999999e-05 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.667803651, 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.662 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:12.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T12:59:12.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.667749743999998, 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.668 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T12:59:12.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T12:59:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.667707439000001, 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:12.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T12:59:12.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.667549013999999, 2024-09-16T12:59:12.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T12:59:12.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:12.686 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.667499533000001, 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T12:59:12.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.692 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.667368963000001, 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T12:59:12.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T12:59:12.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.66718745, 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T12:59:12.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T12:59:12.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T12:59:12.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.856659783000001, 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T12:59:12.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T12:59:12.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T12:59:12.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T12:59:12.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.856510829000001, 2024-09-16T12:59:12.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T12:59:12.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T12:59:12.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:12.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:12.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:12.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:12.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T12:59:12.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T12:59:12.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.717 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:12.717 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:12.717 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:12.719 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:12.715+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:12.719 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T12:59:12.779 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T12:59:12.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T12:59:12.820 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:12.815+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:12.853 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:12.853 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:12.862 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:12.920 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:12.915+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:12.959 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:12.959 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:12.959 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:12.959 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:12.960 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:12.969 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:13.020 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:13.015+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:13.121 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:13.115+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:17 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'7, mlcod=16'5)", 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.161777+0000", 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.966804099000001, 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022199640000000001, 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.161777+0000", 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.161777+0000", 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.161778+0000", 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5370000000000001e-06 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.161792+0000", 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4283e-05 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.161822+0000", 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9133999999999999e-05 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.161834+0000", 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2794e-05 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.161914+0000", 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0018000000000005e-05 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.161957+0000", 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3105999999999997e-05 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.163875+0000", 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001917621 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.163997+0000", 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121471 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-16T12:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.169566+0000", 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.959014920000001, 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033952489999999999, 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169566+0000", 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169566+0000", 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169568+0000", 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8959999999999999e-06 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169803+0000", 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023505499999999999 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169836+0000", 2024-09-16T12:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3283e-05 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169852+0000", 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5812999999999999e-05 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169990+0000", 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138613 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.170035+0000", 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4589000000000003e-05 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.172855+0000", 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028203490000000002 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.172961+0000", 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105651 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:21 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'11, mlcod=16'8)", 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.170604+0000", 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.957976923, 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023969669999999998, 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.170604+0000", 2024-09-16T12:59:13.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.170604+0000", 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.170605+0000", 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.623e-06 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.170765+0000", 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000159507 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.170841+0000", 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6497999999999998e-05 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.170858+0000", 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6563e-05 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.170983+0000", 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012454999999999999 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.171047+0000", 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4373000000000002e-05 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.172968+0000", 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001921318 2024-09-16T12:59:13.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173001+0000", 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2534999999999999e-05 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:22 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'12, mlcod=16'8)", 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.171857+0000", 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.956724050999998, 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020879990000000001, 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.171857+0000", 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.171857+0000", 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.171858+0000", 2024-09-16T12:59:13.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8339999999999999e-06 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.172039+0000", 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000180142 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.172064+0000", 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5434000000000001e-05 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.172078+0000", 2024-09-16T12:59:13.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3926e-05 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.172139+0000", 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1174999999999996e-05 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.172170+0000", 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0412999999999998e-05 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173881+0000", 2024-09-16T12:59:13.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017115120000000001 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173945+0000", 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3563000000000001e-05 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.152 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:23 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'13, mlcod=16'8)", 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.173078+0000", 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.955502837000001, 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024092269999999999, 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173078+0000", 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173078+0000", 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173079+0000", 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.46e-06 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173173+0000", 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3335e-05 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173203+0000", 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0592000000000001e-05 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173215+0000", 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1887e-05 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173267+0000", 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2355000000000003e-05 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.173295+0000", 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.783e-05 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.175357+0000", 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020618680000000001 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.175487+0000", 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012990000000000001 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:43 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'25, mlcod=16'21)", 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.377964+0000", 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.750616312999998, 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020155899999999998, 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.377964+0000", 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.377964+0000", 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.377965+0000", 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0699999999999999e-06 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.377981+0000", 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5155e-05 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.378004+0000", 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3189000000000001e-05 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.378015+0000", 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0993e-05 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.378073+0000", 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8143999999999998e-05 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.378096+0000", 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700999999999998e-05 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.379907+0000", 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018109720000000001 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.379980+0000", 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3366000000000002e-05 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:46 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'28, mlcod=16'24)", 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.381662+0000", 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.746918456, 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027833850000000002, 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.381662+0000", 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.381662+0000", 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.381663+0000", 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0839999999999999e-06 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.381674+0000", 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0994e-05 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.381699+0000", 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5147000000000002e-05 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.381711+0000", 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-05 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.381785+0000", 2024-09-16T12:59:13.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4690999999999994e-05 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.381829+0000", 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3541999999999997e-05 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.384371+0000", 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025421620000000002 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.384446+0000", 2024-09-16T12:59:13.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4574999999999997e-05 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:47 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'29, mlcod=16'24)", 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.382538+0000", 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.746042936999999, 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025219690000000002, 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.382538+0000", 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.382538+0000", 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.382539+0000", 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0860000000000001e-06 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.382616+0000", 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6705999999999994e-05 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.382639+0000", 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3037999999999999e-05 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.382654+0000", 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5837000000000001e-05 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.382742+0000", 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8059000000000005e-05 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.382773+0000", 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0936999999999999e-05 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.385000+0000", 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022267770000000001 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.385060+0000", 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9528999999999999e-05 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:48 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'30, mlcod=16'26)", 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.383711+0000", 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.744869914999999, 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020884459999999999, 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.383711+0000", 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.383711+0000", 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.383712+0000", 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.006e-06 2024-09-16T12:59:13.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.383720+0000", 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1859999999999992e-06 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.383736+0000", 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6421000000000001e-05 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.383746+0000", 2024-09-16T12:59:13.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4760000000000004e-06 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.383854+0000", 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107648 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.383900+0000", 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6434999999999998e-05 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.385746+0000", 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018461129999999999 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.385799+0000", 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3161000000000001e-05 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:49 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'31, mlcod=16'26)", 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.384273+0000", 2024-09-16T12:59:13.173 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.744308123, 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001988044, 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.384273+0000", 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.384273+0000", 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.384274+0000", 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.026e-06 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.384284+0000", 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0492000000000001e-05 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.384303+0000", 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9188999999999999e-05 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.384315+0000", 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1412e-05 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.384538+0000", 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000222971 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.384568+0000", 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0209000000000002e-05 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.386211+0000", 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001642688 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.386261+0000", 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0056999999999999e-05 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:64 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'42, mlcod=16'40)", 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.916925+0000", 2024-09-16T12:59:13.177 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.211655704000002, 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002267781, 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.916925+0000", 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.916925+0000", 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.916927+0000", 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7430000000000001e-06 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.916940+0000", 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2903e-05 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.916973+0000", 2024-09-16T12:59:13.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3386000000000002e-05 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.916985+0000", 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1863e-05 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.917114+0000", 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000129231 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.917160+0000", 2024-09-16T12:59:13.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5652000000000003e-05 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.919079+0000", 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001919118 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.919193+0000", 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113885 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:65 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=16'40)", 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.918099+0000", 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.210481937000001, 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021095580000000001, 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918099+0000", 2024-09-16T12:59:13.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918099+0000", 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918100+0000", 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.64e-06 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918116+0000", 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.541e-05 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918149+0000", 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2722999999999998e-05 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918187+0000", 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8806000000000001e-05 2024-09-16T12:59:13.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918343+0000", 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000155229 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918453+0000", 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010996499999999999 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.920126+0000", 2024-09-16T12:59:13.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016731599999999999 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.920208+0000", 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2625000000000003e-05 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:66 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=16'40)", 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.918767+0000", 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.209813663999999, 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025801830000000002, 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918767+0000", 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918767+0000", 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918768+0000", 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.443e-06 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918898+0000", 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012974699999999999 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918918+0000", 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0216000000000001e-05 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.918928+0000", 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4429999999999992e-06 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.919241+0000", 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031305699999999999 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.919279+0000", 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7919000000000001e-05 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.921243+0000", 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019643769999999998 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.921347+0000", 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010398100000000001 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:67 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=16'40)", 2024-09-16T12:59:13.190 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.920157+0000", 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.208423540999998, 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022382370000000001, 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.920157+0000", 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.920157+0000", 2024-09-16T12:59:13.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.920159+0000", 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5859999999999999e-06 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.920171+0000", 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2243e-05 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.920202+0000", 2024-09-16T12:59:13.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0836000000000001e-05 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.920217+0000", 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5206e-05 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.920314+0000", 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7387999999999997e-05 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.920351+0000", 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6723000000000001e-05 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.922287+0000", 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019358509999999999 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.922395+0000", 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108404 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:68 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=16'41)", 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.921391+0000", 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.207189729, 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024411369999999999, 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.921391+0000", 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.921391+0000", 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.921392+0000", 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4470000000000001e-06 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.921405+0000", 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2683e-05 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.921428+0000", 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3159999999999998e-05 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.921439+0000", 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1066e-05 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.921548+0000", 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010894400000000001 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.921584+0000", 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5366e-05 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.923710+0000", 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021267170000000002 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.923832+0000", 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121754 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.198 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:69 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'42)", 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.922443+0000", 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.206137304999999, 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025085120000000001, 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.922443+0000", 2024-09-16T12:59:13.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.922443+0000", 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.922445+0000", 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.406e-06 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.922496+0000", 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1273000000000002e-05 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.922524+0000", 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7478999999999999e-05 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.922534+0000", 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0753e-05 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.922636+0000", 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101431 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.922688+0000", 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1993000000000002e-05 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.924846+0000", 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021578130000000002 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.924952+0000", 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106364 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:72 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'45)", 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.926792+0000", 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.201788397000001, 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024641899999999998, 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.926792+0000", 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.926792+0000", 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.926794+0000", 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6440000000000001e-06 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.926949+0000", 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015469 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.926978+0000", 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9731999999999999e-05 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.926990+0000", 2024-09-16T12:59:13.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1333e-05 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.927071+0000", 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1588999999999994e-05 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.927105+0000", 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3912999999999999e-05 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.929144+0000", 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020386950000000001 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.929256+0000", 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112594 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:88 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'62, mlcod=17'60)", 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:47.728770+0000", 2024-09-16T12:59:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.399810287000001, 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022078480000000001, 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.728770+0000", 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.728770+0000", 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.728772+0000", 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.637e-06 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.728791+0000", 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9077e-05 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.728824+0000", 2024-09-16T12:59:13.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2388999999999998e-05 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.728838+0000", 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4633e-05 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.728989+0000", 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150956 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.729052+0000", 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2772000000000001e-05 2024-09-16T12:59:13.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.730858+0000", 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806392 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.730978+0000", 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119992 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.212 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:90 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'64, mlcod=17'60)", 2024-09-16T12:59:13.212 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:47.731429+0000", 2024-09-16T12:59:13.212 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.397152008999999, 2024-09-16T12:59:13.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002089259, 2024-09-16T12:59:13.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.212 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.731429+0000", 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.731429+0000", 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.731430+0000", 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.643e-06 2024-09-16T12:59:13.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.731600+0000", 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000169502 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.731629+0000", 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9442e-05 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.731645+0000", 2024-09-16T12:59:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5800000000000001e-05 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.731831+0000", 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018546400000000001 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.731863+0000", 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2634e-05 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.733470+0000", 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016064779999999999 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.733518+0000", 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8296e-05 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:95 1.0 e17/15 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-16T12:59:13.216 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:47.743548+0000", 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.385032783, 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020492599999999998, 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.743548+0000", 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.743548+0000", 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.743549+0000", 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3629999999999999e-06 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.743613+0000", 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3460999999999994e-05 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.743636+0000", 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3337e-05 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.743645+0000", 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6750000000000008e-06 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.743729+0000", 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4086999999999996e-05 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.743768+0000", 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9075000000000001e-05 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.745507+0000", 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001739324 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:47.745597+0000", 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9938000000000005e-05 2024-09-16T12:59:13.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.221 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:13.222 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:13.222 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:13.215+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:13.323 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:13.319+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:13.372 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:13.372 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:13.372 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:13.372 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:13.382 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:13.424 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:13.419+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:13.479 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:13.480 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:13.480 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:13.480 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:13.480 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:13.489 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:13.525 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:13.519+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:13.596 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:13.596 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4272.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:54.643209+0000", 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.94941562, 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016437139999999999, 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:54.643209+0000", 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:54.643209+0000", 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:54.643210+0000", 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9100000000000003e-07 2024-09-16T12:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:54.643267+0000", 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6576000000000002e-05 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:54.643319+0000", 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2423999999999998e-05 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:54.643334+0000", 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4815e-05 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:54.643428+0000", 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4110000000000005e-05 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:54.643487+0000", 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9555999999999997e-05 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:54.644800+0000", 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001312269 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:54.644853+0000", 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3273000000000003e-05 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4272.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-16T12:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:55.650848+0000", 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.941776684000001, 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011666440000000001, 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:55.650848+0000", 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:55.650848+0000", 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:55.650849+0000", 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3599999999999999e-07 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:55.650859+0000", 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0156000000000001e-05 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:55.650894+0000", 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4684999999999997e-05 2024-09-16T12:59:13.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:55.650911+0000", 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689e-05 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:55.650984+0000", 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2471000000000005e-05 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:55.651016+0000", 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2434999999999997e-05 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:55.651967+0000", 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00095085299999999998 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:55.652015+0000", 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7919e-05 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:13.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:13.606 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:13.607 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:13.625 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:13.619+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:13.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:13.759 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:13.755+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:13.780 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T12:59:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T12:59:13.794 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T12:59:13.826 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:13.819+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:13.841 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:13.841 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:13.855 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:13.927 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:13.923+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:14.015 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:14.015 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:14.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:14.015 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:14.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.015 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4272.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T12:59:14.015 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:53.630084+0000", 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.380425579000001, 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093404800000000004, 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630084+0000", 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630084+0000", 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630084+0000", 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5000000000000002e-07 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630092+0000", 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.994e-06 2024-09-16T12:59:14.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630110+0000", 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7957000000000001e-05 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630121+0000", 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0206e-05 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:14.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630167+0000", 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6238e-05 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630204+0000", 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7459999999999997e-05 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630960+0000", 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00075580500000000002 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.631018+0000", 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7637999999999999e-05 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.23085450099999999, 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003284908, 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 10 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:14.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779657+0000", 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1239999999999999e-06 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779667+0000", 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0487e-05 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779683+0000", 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6146000000000001e-05 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779695+0000", 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1347e-05 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779769+0000", 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.449e-05 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779918+0000", 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000148462 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.780088+0000", 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017066600000000001 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:14.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.781019+0000", 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093024399999999999 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782840+0000", 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018211379999999999 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782899+0000", 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8777000000000003e-05 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782940+0000", 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1026999999999999e-05 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.782710+0000", 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.22779906599999999, 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023676400000000001, 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:14.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782710+0000", 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782710+0000", 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782712+0000", 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1100000000000001e-06 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782722+0000", 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4460000000000007e-06 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782747+0000", 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5130000000000002e-05 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782756+0000", 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3190000000000001e-06 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782822+0000", 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6313e-05 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782831+0000", 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9309999999999997e-06 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:14.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782947+0000", 2024-09-16T12:59:14.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011551499999999999 2024-09-16T12:59:14.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.030 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:14.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.030 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:14.030 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:14.031 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:14.032 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:14.027+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:14.133 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:14.127+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:14.147 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:14.233 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:14.227+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:14.249 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:14.328 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:14.334 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:14.331+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:7 4.1a e33/31 4:5cea7035:::rbd_info:head v 33'1, mlcod=0'0)", 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:12.762669+0000", 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.5818427530000001, 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097892500000000006, 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:14.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:14.350 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:14.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:14.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762669+0000", 2024-09-16T12:59:14.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:14.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762669+0000", 2024-09-16T12:59:14.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762670+0000", 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2800000000000003e-07 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762679+0000", 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9889999999999998e-06 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762704+0000", 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5754000000000001e-05 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762716+0000", 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1664e-05 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762769+0000", 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3059000000000001e-05 2024-09-16T12:59:14.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762788+0000", 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9008000000000002e-05 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.763601+0000", 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00081281500000000004 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.763648+0000", 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7308000000000002e-05 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:9 4.1a e34/31 4:5cea7035:::rbd_info:head v 34'2, mlcod=0'0)", 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.775985+0000", 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.56852754999999999, 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099334699999999989, 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:14.354 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.775985+0000", 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.775985+0000", 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.775985+0000", 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.775995+0000", 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0051000000000001e-05 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776012+0000", 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7193999999999998e-05 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776027+0000", 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4676e-05 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776093+0000", 2024-09-16T12:59:14.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6118000000000006e-05 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776110+0000", 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7516999999999998e-05 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776932+0000", 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00082138199999999995 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776978+0000", 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6112999999999997e-05 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:12 4.c e34/31 4:35454ec4:::rbd_header.11b524d5182b:head v 34'1, mlcod=0'0)", 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.793656+0000", 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.55085622899999998, 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001067599, 2024-09-16T12:59:14.359 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793656+0000", 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793656+0000", 2024-09-16T12:59:14.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793658+0000", 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.829e-06 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793675+0000", 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:14.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793705+0000", 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0040999999999999e-05 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793717+0000", 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1474999999999999e-05 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793785+0000", 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8322000000000001e-05 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793815+0000", 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9584e-05 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.794668+0000", 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085372699999999998 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.794724+0000", 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5232000000000002e-05 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.363 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:25 4.c e34/31 4:35454ec4:::rbd_header.11b524d5182b:head v 34'2, mlcod=0'0)", 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.814188+0000", 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.53032438800000004, 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010709739999999999, 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:14.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814188+0000", 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814188+0000", 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814190+0000", 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0439999999999998e-06 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814206+0000", 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6578e-05 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814240+0000", 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3183999999999999e-05 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814255+0000", 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5307000000000001e-05 2024-09-16T12:59:14.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814337+0000", 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2311000000000005e-05 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814378+0000", 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0519000000000003e-05 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.815171+0000", 2024-09-16T12:59:14.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00079328999999999999 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.815259+0000", 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7740999999999999e-05 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:14.369 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:14.438 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:14.435+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:14.461 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:14.478 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:14.478 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:14.478 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:14.478 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:14.481 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:14.488 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:14.507 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:14.526 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:14.539 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:14.535+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:14.549 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:14.572 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:14.589 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:14.589 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:14.594 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:14.599 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:14.620 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:14.622 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:14.622 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:14.622 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:14.622 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:14.622 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:14.622 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:14.623 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:14.623 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:14.640 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:14.635+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:14.694 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:14.694 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:14.703 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:14.741 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:14.735+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:14.783 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T12:59:14.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T12:59:14.806 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:14.806 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:14.816 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:14.859 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:14.855+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:14.942 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:14.935+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:14.960 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:14.960 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:14.969 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:15.043 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:15.039+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:15.072 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:15.073 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:15.082 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:15.145 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:15.139+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:15.246 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:15.239+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:15.248 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:15.249 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:15.258 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:15.346 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:15.339+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:15.361 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:15.361 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:15.370 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:15.448 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:15.443+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:15.470 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:15.470 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:15.479 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:15.548 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:15.543+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:15.578 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:15.578 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:15.587 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:15.650 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:15.643+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:15.687 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:15.687 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:15.697 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:15.751 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:15.747+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:15.794 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T12:59:15.798 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:15.798 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:15.799 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T12:59:15.807 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:15.813 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T12:59:15.851 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:15.847+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:15.868 INFO:teuthology.orchestra.run.smithi042.stdout:skipping zero size read 2024-09-16T12:59:15.868 INFO:teuthology.orchestra.run.smithi042.stdout:skipping zero length punch hole 2024-09-16T12:59:15.868 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xdfc8230 2024-09-16T12:59:15.887 INFO:teuthology.orchestra.run.smithi042.stdout:9 trunc from 0x0 to 0xdfc8230 2024-09-16T12:59:15.901 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:15.901 INFO:teuthology.orchestra.run.smithi042.stdout:12 write 0xcba1257 thru 0xcba86ac (0x7456 bytes) 2024-09-16T12:59:15.907 INFO:teuthology.orchestra.run.smithi042.stdout:13 write 0x4d79033 thru 0x4d8523f (0xc20d bytes) 2024-09-16T12:59:15.915 INFO:teuthology.orchestra.run.smithi042.stdout:14 read 0xbd8e0f0 thru 0xbd9718e (0x909f bytes) 2024-09-16T12:59:15.926 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:15.926 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:15.929 INFO:teuthology.orchestra.run.smithi042.stdout:15 trunc from 0xdfc8230 to 0x31072da 2024-09-16T12:59:15.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:15.952 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:15.947+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:15.953 INFO:teuthology.orchestra.run.smithi042.stdout:16 write 0x2abe735 thru 0x2aca85f (0xc12b bytes) 2024-09-16T12:59:16.027 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:16.027 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:16.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:16.052 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:16.047+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:16.131 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:16.131 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:16.140 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:16.160 INFO:teuthology.orchestra.run.smithi042.stdout:18 punch from 0xe3a154 to 0xe49fdf, (0xfe8b bytes) 2024-09-16T12:59:16.160 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:16.160 INFO:teuthology.orchestra.run.smithi042.stdout:20 write 0xa912c9 thru 0xa99c60 (0x8998 bytes) 2024-09-16T12:59:16.161 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:16.155+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:22 write 0x1f4195a thru 0x1f48750 (0x6df7 bytes) 2024-09-16T12:59:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:23 read 0x2b78848 thru 0x2b7f002 (0x67bb bytes) 2024-09-16T12:59:16.202 INFO:teuthology.orchestra.run.smithi042.stdout:25 writesame 0xb467278 thru 0xb47546d data_size 0x1f(0xe1f6 bytes) 2024-09-16T12:59:16.202 INFO:teuthology.orchestra.run.smithi042.stdout:27 write 0xa43bde9 thru 0xa44897a (0xcb92 bytes) 2024-09-16T12:59:16.209 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:16.209 INFO:teuthology.orchestra.run.smithi042.stdout:30 read 0x46edc1c thru 0x46f5a06 (0x7deb bytes) 2024-09-16T12:59:16.220 INFO:teuthology.orchestra.run.smithi042.stdout:32 writesame 0xb6a71bc thru 0xb6ad674 data_size 0x2d(0x64b9 bytes) 2024-09-16T12:59:16.235 INFO:teuthology.orchestra.run.smithi042.stdout:33 write 0xd061961 thru 0xd07158e (0xfc2e bytes) 2024-09-16T12:59:16.241 INFO:teuthology.orchestra.run.smithi042.stdout:34 writesame 0x423567a thru 0x4236149 data_size 0x8(0xad0 bytes) 2024-09-16T12:59:16.248 INFO:teuthology.orchestra.run.smithi042.stdout:35 writesame 0x23e8cc2 thru 0x23f7ff8 data_size 0x1d(0xf337 bytes) 2024-09-16T12:59:16.254 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:16.247+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:16.257 INFO:teuthology.orchestra.run.smithi042.stdout:37 write 0x1aef243 thru 0x1aef8d6 (0x694 bytes) 2024-09-16T12:59:16.261 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:16.261 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:16.264 INFO:teuthology.orchestra.run.smithi042.stdout:38 writesame 0x7e58506 thru 0x7e5f7c2 data_size 0x1(0x72bd bytes) 2024-09-16T12:59:16.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:16.293 INFO:teuthology.orchestra.run.smithi042.stdout:39 read 0x92fd72c thru 0x930d69b (0xff70 bytes) 2024-09-16T12:59:16.305 INFO:teuthology.orchestra.run.smithi042.stdout:40 trunc from 0xd07158f to 0x624ffbd 2024-09-16T12:59:16.329 INFO:teuthology.orchestra.run.smithi042.stdout:41 write 0x47a7a21 thru 0x47b6510 (0xeaf0 bytes) 2024-09-16T12:59:16.348 INFO:teuthology.orchestra.run.smithi042.stdout:42 writesame 0x757b0cd thru 0x758308c data_size 0x70(0x7fc0 bytes) 2024-09-16T12:59:16.348 INFO:teuthology.orchestra.run.smithi042.stdout:43 punch from 0x566e4ed to 0x5672d09, (0x481c bytes) 2024-09-16T12:59:16.348 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:16.355 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:16.351+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:16.367 INFO:teuthology.orchestra.run.smithi042.stdout:45 writesame 0xbe10f6a thru 0xbe147cb data_size 0x7(0x3862 bytes) 2024-09-16T12:59:16.367 INFO:teuthology.orchestra.run.smithi042.stdout:46 writesame 0x81eb179 thru 0x81f4c8b data_size 0xb(0x9b13 bytes) 2024-09-16T12:59:16.374 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:16.374 INFO:teuthology.orchestra.run.smithi042.stdout:49 read 0xba1ba4f thru 0xba29c0d (0xe1bf bytes) 2024-09-16T12:59:16.374 INFO:teuthology.orchestra.run.smithi042.stdout:50 write 0x3ae3247 thru 0x3aed236 (0x9ff0 bytes) 2024-09-16T12:59:16.381 INFO:teuthology.orchestra.run.smithi042.stdout:51 read 0x5475f3 thru 0x55443f (0xce4d bytes) 2024-09-16T12:59:16.381 INFO:teuthology.orchestra.run.smithi042.stdout:52 punch from 0x993d7a1 to 0x993f678, (0x1ed7 bytes) 2024-09-16T12:59:16.381 INFO:teuthology.orchestra.run.smithi042.stdout:53 write 0x3d98c7b thru 0x3da5da7 (0xd12d bytes) 2024-09-16T12:59:16.389 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:16.389 INFO:teuthology.orchestra.run.smithi042.stdout:55 read 0x53ec889 thru 0x53f7ec1 (0xb639 bytes) 2024-09-16T12:59:16.401 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:16.401 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:16.407 INFO:teuthology.orchestra.run.smithi042.stdout:56 write 0xe217786 thru 0xe220ee2 (0x975d bytes) 2024-09-16T12:59:16.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:58 read 0x8777347 thru 0x8786a15 (0xf6cf bytes) 2024-09-16T12:59:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:59 read 0x871a872 thru 0x8727743 (0xced2 bytes) 2024-09-16T12:59:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:62 punch from 0xd2153cb to 0xd22515c, (0xfd91 bytes) 2024-09-16T12:59:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:63 write 0xcd1cfc2 thru 0xcd2c5bc (0xf5fb bytes) 2024-09-16T12:59:16.422 INFO:teuthology.orchestra.run.smithi042.stdout:66 read 0xe04e721 thru 0xe05890d (0xa1ed bytes) 2024-09-16T12:59:16.424 INFO:teuthology.orchestra.run.smithi042.stdout:68 read 0x657d7d7 thru 0x657db61 (0x38b bytes) 2024-09-16T12:59:16.424 INFO:teuthology.orchestra.run.smithi042.stdout:69 punch from 0xaead292 to 0xaeae05c, (0xdca bytes) 2024-09-16T12:59:16.425 INFO:teuthology.orchestra.run.smithi042.stdout:70 writesame 0x1ee1dfc thru 0x1eefb93 data_size 0x1c(0xdd98 bytes) 2024-09-16T12:59:16.428 INFO:teuthology.orchestra.run.smithi042.stdout:71 read 0x7e35c40 thru 0x7e425e7 (0xc9a8 bytes) 2024-09-16T12:59:16.428 INFO:teuthology.orchestra.run.smithi042.stdout:74 write 0x59a7cdd thru 0x59b1f95 (0xa2b9 bytes) 2024-09-16T12:59:16.435 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:16.435 INFO:teuthology.orchestra.run.smithi042.stdout:76 write 0xc71a94a thru 0xc71fe48 (0x54ff bytes) 2024-09-16T12:59:16.441 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:16.441 INFO:teuthology.orchestra.run.smithi042.stdout:78 read 0x959ff2d thru 0x95a575f (0x5833 bytes) 2024-09-16T12:59:16.441 INFO:teuthology.orchestra.run.smithi042.stdout:80 writesame 0x537c7f7 thru 0x5385321 data_size 0x1(0x8b2b bytes) 2024-09-16T12:59:16.456 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:16.451+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:16.475 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:16.483 INFO:teuthology.orchestra.run.smithi042.stdout:82 trunc from 0xe220ee3 to 0xa4246cd 2024-09-16T12:59:16.516 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:16.516 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:16.518 INFO:teuthology.orchestra.run.smithi042.stdout:83 trunc from 0xa4246cd to 0x3890f29 2024-09-16T12:59:16.525 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:16.555 INFO:teuthology.orchestra.run.smithi042.stdout:84 trunc from 0x3890f29 to 0x8e14395 2024-09-16T12:59:16.557 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:16.551+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:85 trunc from 0x8e14395 to 0x17251e7 2024-09-16T12:59:16.616 INFO:teuthology.orchestra.run.smithi042.stdout:89 write 0xa48e26 thru 0xa517e9 (0x89c4 bytes) 2024-09-16T12:59:16.620 INFO:teuthology.orchestra.run.smithi042.stdout:91 punch from 0x12ff23a to 0x1300986, (0x174c bytes) 2024-09-16T12:59:16.620 INFO:teuthology.orchestra.run.smithi042.stdout:92 punch from 0x161bd9c to 0x161c716, (0x97a bytes) 2024-09-16T12:59:16.637 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:16.651 INFO:teuthology.orchestra.run.smithi042.stdout:93 writesame 0xb36ede1 thru 0xb37b10e data_size 0x56(0xc32e bytes) 2024-09-16T12:59:16.651 INFO:teuthology.orchestra.run.smithi042.stdout:94 write 0x4e1d16b thru 0x4e21d54 (0x4bea bytes) 2024-09-16T12:59:16.657 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:16.651+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:16.665 INFO:teuthology.orchestra.run.smithi042.stdout:96 trunc from 0xb37b10f to 0x749ee90 2024-09-16T12:59:16.667 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:16.667 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:16.676 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:16.678 INFO:teuthology.orchestra.run.smithi042.stdout:98 write 0x6549647 thru 0x6556529 (0xcee3 bytes) 2024-09-16T12:59:16.686 INFO:teuthology.orchestra.run.smithi042.stdout:99 read 0x514e715 thru 0x5150f87 (0x2873 bytes) 2024-09-16T12:59:16.686 INFO:teuthology.orchestra.run.smithi042.stdout:101 writesame 0x93e3ed thru 0x948c76 data_size 0x66(0xa88a bytes) 2024-09-16T12:59:16.689 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:16.710 INFO:teuthology.orchestra.run.smithi042.stdout:103 write 0xd2e45c1 thru 0xd2ef138 (0xab78 bytes) 2024-09-16T12:59:16.735 INFO:teuthology.orchestra.run.smithi042.stdout:104 trunc from 0xd2ef139 to 0x6f794 2024-09-16T12:59:16.758 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:16.755+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:16.768 INFO:teuthology.orchestra.run.smithi042.stdout:105 writesame 0x6e21e5 thru 0x6eed25 data_size 0x1(0xcb41 bytes) 2024-09-16T12:59:16.769 INFO:teuthology.orchestra.run.smithi042.stdout:106 read 0x2125f3 thru 0x21bef6 (0x9904 bytes) 2024-09-16T12:59:16.769 INFO:teuthology.orchestra.run.smithi042.stdout:107 clone 1 order 20 su 65536 sc 2 2024-09-16T12:59:16.775 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:16.775 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:16.784 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:16.807 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T12:59:16.808 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T12:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:16.821 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T12:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:16.859 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:16.855+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:16.871 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:16.880 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:16.880 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:16.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:16.880 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:16.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4272.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:53.630084+0000", 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.246113292, 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093404800000000004, 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630084+0000", 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630084+0000", 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630084+0000", 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5000000000000002e-07 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630092+0000", 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.994e-06 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:16.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630110+0000", 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7957000000000001e-05 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630121+0000", 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0206e-05 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630167+0000", 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6238e-05 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:16.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630204+0000", 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7459999999999997e-05 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.630960+0000", 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00075580500000000002 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:53.631018+0000", 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7637999999999999e-05 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:16.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.0965422139999998, 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003284908, 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 10 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:16.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779657+0000", 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1239999999999999e-06 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779667+0000", 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0487e-05 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779683+0000", 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6146000000000001e-05 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779695+0000", 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1347e-05 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779769+0000", 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.449e-05 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779918+0000", 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000148462 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.780088+0000", 2024-09-16T12:59:16.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017066600000000001 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.781019+0000", 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093024399999999999 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782840+0000", 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018211379999999999 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782899+0000", 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8777000000000003e-05 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782940+0000", 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1026999999999999e-05 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T12:59:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.782710+0000", 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.093486779, 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023676400000000001, 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782710+0000", 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782710+0000", 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782712+0000", 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1100000000000001e-06 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782722+0000", 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4460000000000007e-06 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782747+0000", 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5130000000000002e-05 2024-09-16T12:59:16.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782756+0000", 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3190000000000001e-06 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782822+0000", 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6313e-05 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782831+0000", 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9309999999999997e-06 2024-09-16T12:59:16.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:16.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:16.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:16.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782947+0000", 2024-09-16T12:59:16.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011551499999999999 2024-09-16T12:59:16.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:16.894 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:16.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:16.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:16.894 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:16.894 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:16.895 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:16.895 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:16.916 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:16.939 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:16.942 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:16.946 INFO:teuthology.orchestra.run.smithi042.stdout:108 read 0x13b88 thru 0x181b4 (0x462d bytes) 2024-09-16T12:59:16.947 INFO:teuthology.orchestra.run.smithi042.stdout:109 read 0x55f137 thru 0x5663d4 (0x729e bytes) 2024-09-16T12:59:16.960 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:16.955+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:16.969 INFO:teuthology.orchestra.run.smithi042.stdout:110 write 0x1fc9a20 thru 0x1fcfb27 (0x6108 bytes) 2024-09-16T12:59:16.976 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xe5e835f 2024-09-16T12:59:16.993 INFO:teuthology.orchestra.run.smithi042.stdout:111 trunc from 0x1fcfb28 to 0xe5e835f 2024-09-16T12:59:17.006 INFO:teuthology.orchestra.run.smithi042.stdout:112 write 0xd204921 thru 0xd213c44 (0xf324 bytes) 2024-09-16T12:59:17.015 INFO:teuthology.orchestra.run.smithi042.stdout:113 read 0xb1bd58 thru 0xb1de3b (0x20e4 bytes) 2024-09-16T12:59:17.016 INFO:teuthology.orchestra.run.smithi042.stdout:115 write 0xaa89a6a thru 0xaa90c3d (0x71d4 bytes) 2024-09-16T12:59:17.023 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:17.023 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:17.024 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.038 INFO:teuthology.orchestra.run.smithi042.stdout:120 trunc from 0xe5e835f to 0x25c5e32 2024-09-16T12:59:17.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:17.066 INFO:teuthology.orchestra.run.smithi042.stdout:123 trunc from 0x25c5e32 to 0x7457a7b 2024-09-16T12:59:17.067 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:17.059+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:17.077 INFO:teuthology.orchestra.run.smithi042.stdout:124 punch from 0x215db7d to 0x216d481, (0xf904 bytes) 2024-09-16T12:59:17.078 INFO:teuthology.orchestra.run.smithi042.stdout:131 read 0x40d3424 thru 0x40d4241 (0xe1e bytes) 2024-09-16T12:59:17.097 INFO:teuthology.orchestra.run.smithi042.stdout:133 write 0xa58bcf0 thru 0xa59bbeb (0xfefc bytes) 2024-09-16T12:59:17.105 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.105 INFO:teuthology.orchestra.run.smithi042.stdout:136 writesame 0x16798fe thru 0x1685867 data_size 0x3(0xbf6a bytes) 2024-09-16T12:59:17.117 INFO:teuthology.orchestra.run.smithi042.stdout:137 read 0x1636efa thru 0x163b33c (0x4443 bytes) 2024-09-16T12:59:17.118 INFO:teuthology.orchestra.run.smithi042.stdout:138 read 0x9dd7067 thru 0x9de2644 (0xb5de bytes) 2024-09-16T12:59:17.118 INFO:teuthology.orchestra.run.smithi042.stdout:139 write 0x8d92eeb thru 0x8da2e85 (0xff9b bytes) 2024-09-16T12:59:17.126 INFO:teuthology.orchestra.run.smithi042.stdout:140 read 0x6b7161d thru 0x6b71776 (0x15a bytes) 2024-09-16T12:59:17.143 INFO:teuthology.orchestra.run.smithi042.stdout:141 write 0xdd613f6 thru 0xdd61c3f (0x84a bytes) 2024-09-16T12:59:17.149 INFO:teuthology.orchestra.run.smithi042.stdout:142 read 0x25d09cd thru 0x25d4da5 (0x43d9 bytes) 2024-09-16T12:59:17.149 INFO:teuthology.orchestra.run.smithi042.stdout:143 write 0x444ccd9 thru 0x445cc21 (0xff49 bytes) 2024-09-16T12:59:17.157 INFO:teuthology.orchestra.run.smithi042.stdout:144 punch from 0xa9843cb to 0xa986cda, (0x290f bytes) 2024-09-16T12:59:17.158 INFO:teuthology.orchestra.run.smithi042.stdout:145 read 0xaaafcae thru 0xaab1b40 (0x1e93 bytes) 2024-09-16T12:59:17.158 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.162 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:17.155+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:17.172 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:17.172 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:17.172 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:17.172 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:17.172 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:17.177 INFO:teuthology.orchestra.run.smithi042.stdout:147 trunc from 0xdd61c40 to 0x80b14 2024-09-16T12:59:17.185 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:17.234 INFO:teuthology.orchestra.run.smithi042.stdout:148 write 0x4ca9cd3 thru 0x4cb7080 (0xd3ae bytes) 2024-09-16T12:59:17.244 INFO:teuthology.orchestra.run.smithi042.stdout:150 read 0x4201efe thru 0x42092c8 (0x73cb bytes) 2024-09-16T12:59:17.244 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.245 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.245 INFO:teuthology.orchestra.run.smithi042.stdout:154 punch from 0x2fdec20 to 0x2feaa49, (0xbe29 bytes) 2024-09-16T12:59:17.245 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.245 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.263 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:17.259+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:17.263 INFO:teuthology.orchestra.run.smithi042.stdout:157 write 0x7563bd1 thru 0x757174d (0xdb7d bytes) 2024-09-16T12:59:17.272 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.272 INFO:teuthology.orchestra.run.smithi042.stdout:159 read 0x1f465ec thru 0x1f4f381 (0x8d96 bytes) 2024-09-16T12:59:17.272 INFO:teuthology.orchestra.run.smithi042.stdout:161 read 0x22842a1 thru 0x2289cc2 (0x5a22 bytes) 2024-09-16T12:59:17.272 INFO:teuthology.orchestra.run.smithi042.stdout:162 writesame 0x65c057 thru 0x65d0e8 data_size 0x15(0x1092 bytes) 2024-09-16T12:59:17.278 INFO:teuthology.orchestra.run.smithi042.stdout:163 punch from 0x56e16c2 to 0x56f0192, (0xead0 bytes) 2024-09-16T12:59:17.290 INFO:teuthology.orchestra.run.smithi042.stdout:164 write 0x833b779 thru 0x833be63 (0x6eb bytes) 2024-09-16T12:59:17.298 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.298 INFO:teuthology.orchestra.run.smithi042.stdout:167 punch from 0x3d7b055 to 0x3d844a8, (0x9453 bytes) 2024-09-16T12:59:17.298 INFO:teuthology.orchestra.run.smithi042.stdout:168 read 0x3f88b24 thru 0x3f9453e (0xba1b bytes) 2024-09-16T12:59:17.298 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.298 INFO:teuthology.orchestra.run.smithi042.stdout:174 read 0x2c28dc thru 0x2c2dc2 (0x4e7 bytes) 2024-09-16T12:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout:176 trunc from 0x833be64 to 0x3474d61 2024-09-16T12:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout:178 read 0x24003bb thru 0x2402927 (0x256d bytes) 2024-09-16T12:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout:182 read 0x76c90 thru 0x794f5 (0x2866 bytes) 2024-09-16T12:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout:184 read 0xba6bd0 thru 0xbad118 (0x6549 bytes) 2024-09-16T12:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout:186 writesame 0x1f44746 thru 0x1f50680 data_size 0x1(0xbf3b bytes) 2024-09-16T12:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:7 4.1a e33/31 4:5cea7035:::rbd_info:head v 33'1, mlcod=0'0)", 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:12.762669+0000", 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5732757319999999, 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097892500000000006, 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762669+0000", 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762669+0000", 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762670+0000", 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2800000000000003e-07 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762679+0000", 2024-09-16T12:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9889999999999998e-06 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762704+0000", 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5754000000000001e-05 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762716+0000", 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1664e-05 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762769+0000", 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3059000000000001e-05 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.762788+0000", 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9008000000000002e-05 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.763601+0000", 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00081281500000000004 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:12.763648+0000", 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7308000000000002e-05 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:9 4.1a e34/31 4:5cea7035:::rbd_info:head v 34'2, mlcod=0'0)", 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.775985+0000", 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.559960529, 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099334699999999989, 2024-09-16T12:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.775985+0000", 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.775985+0000", 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.775985+0000", 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.775995+0000", 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0051000000000001e-05 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776012+0000", 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7193999999999998e-05 2024-09-16T12:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776027+0000", 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4676e-05 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776093+0000", 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6118000000000006e-05 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776110+0000", 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7516999999999998e-05 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776932+0000", 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00082138199999999995 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.776978+0000", 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6112999999999997e-05 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:12 4.c e34/31 4:35454ec4:::rbd_header.11b524d5182b:head v 34'1, mlcod=0'0)", 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.793656+0000", 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5422892080000001, 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001067599, 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793656+0000", 2024-09-16T12:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793656+0000", 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793658+0000", 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.829e-06 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793675+0000", 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793705+0000", 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0040999999999999e-05 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793717+0000", 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1474999999999999e-05 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793785+0000", 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8322000000000001e-05 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.793815+0000", 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9584e-05 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.794668+0000", 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085372699999999998 2024-09-16T12:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.794724+0000", 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5232000000000002e-05 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:25 4.c e34/31 4:35454ec4:::rbd_header.11b524d5182b:head v 34'2, mlcod=0'0)", 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.814188+0000", 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5217573670000002, 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010709739999999999, 2024-09-16T12:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814188+0000", 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814188+0000", 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814190+0000", 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0439999999999998e-06 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814206+0000", 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6578e-05 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814240+0000", 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3183999999999999e-05 2024-09-16T12:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814255+0000", 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5307000000000001e-05 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814337+0000", 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2311000000000005e-05 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.814378+0000", 2024-09-16T12:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0519000000000003e-05 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.815171+0000", 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00079328999999999999 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.815259+0000", 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7740999999999999e-05 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:380 4.c e37/31 4:35454ec4:::rbd_header.11b524d5182b:head v 37'29, mlcod=37'27)", 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:16.912237+0000", 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.42370802499999999, 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00081201299999999999, 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.912237+0000", 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.912237+0000", 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.912239+0000", 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4980000000000001e-06 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.912249+0000", 2024-09-16T12:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0302000000000001e-05 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.912269+0000", 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0404e-05 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.912277+0000", 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5460000000000003e-06 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.912363+0000", 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6613999999999994e-05 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.912425+0000", 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1687999999999996e-05 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.913008+0000", 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058287200000000001 2024-09-16T12:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.913049+0000", 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1088999999999999e-05 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:440 4.16 e37/31 4:6ccd461c:::rbd_data.11b59d02cc38.00000000000000d2:head v 37'9, mlcod=36'7)", 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.007775+0000", 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.32816973999999999, 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022675270000000001, 2024-09-16T12:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007775+0000", 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007775+0000", 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007776+0000", 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007822+0000", 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6090000000000001e-05 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007836+0000", 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4326e-05 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007844+0000", 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3900000000000004e-06 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007987+0000", 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014301 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008051+0000", 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4839999999999996e-05 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.009964+0000", 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019124159999999999 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.010043+0000", 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9123999999999997e-05 2024-09-16T12:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:441 4.0 4:07ec4131:::rbd_data.11b59d02cc38.00000000000000d3:head [set-alloc-hint object_size 1048576 write_size 1048576,write 0~15429] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.008323+0000", 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.32762223600000001, 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039226770000000003, 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 441 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008323+0000", 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008323+0000", 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008326+0000", 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.841e-06 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008380+0000", 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4138999999999998e-05 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008431+0000", 2024-09-16T12:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1532000000000001e-05 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008483+0000", 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2077000000000003e-05 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008636+0000", 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015205000000000001 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008927+0000", 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029145199999999999 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.009454+0000", 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000526999 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.011277+0000", 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018226480000000001 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.012144+0000", 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086761499999999997 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.012196+0000", 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1437999999999998e-05 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.012246+0000", 2024-09-16T12:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9886000000000003e-05 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:450 4.0 4:07ec4131:::rbd_data.11b59d02cc38.00000000000000d3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.039373+0000", 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.29657228099999999, 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00227999, 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 450 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.039373+0000", 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.039373+0000", 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.039375+0000", 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.12e-06 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.039383+0000", 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0309999999999998e-06 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.039412+0000", 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9442e-05 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.039421+0000", 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7129999999999994e-06 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.039517+0000", 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5835000000000007e-05 2024-09-16T12:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.039645+0000", 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012784499999999999 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.039801+0000", 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015611999999999999 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.040493+0000", 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00069183999999999999 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.041534+0000", 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010415369999999999 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.041606+0000", 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1662000000000006e-05 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.041653+0000", 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6845000000000003e-05 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:481 4.1a e37/31 4:5f743d4b:::rbd_data.11b59d02cc38.00000000000000a4:head v 37'10, mlcod=36'8)", 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.098561+0000", 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.237384184, 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001387854, 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.098561+0000", 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.098561+0000", 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.098563+0000", 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.098600+0000", 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6820000000000003e-05 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.098621+0000", 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0979999999999999e-05 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.098630+0000", 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0869999999999996e-06 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.098689+0000", 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8906000000000001e-05 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.098717+0000", 2024-09-16T12:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8002999999999999e-05 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.099884+0000", 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001167417 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.099949+0000", 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4672999999999996e-05 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:491 4.19 4:9af48b0e:::rbd_data.11b59d02cc38.000000000000008d:head [set-alloc-hint object_size 1048576 write_size 1048576,write 798443~53525] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.119977+0000", 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.215968569, 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030364810000000002, 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 491 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.119977+0000", 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.119977+0000", 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.119979+0000", 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.588e-06 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.120053+0000", 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4272999999999999e-05 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.120086+0000", 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2369999999999997e-05 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.120098+0000", 2024-09-16T12:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2357999999999999e-05 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.120164+0000", 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6056000000000006e-05 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.120349+0000", 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018515599999999999 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.120514+0000", 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000164525 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.121689+0000", 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011748240000000001 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.122911+0000", 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001222276 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.122967+0000", 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6409000000000002e-05 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.123013+0000", 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5646000000000003e-05 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:505 4.0 4:05d10a49:::rbd_data.11b59d02cc38.0000000000000044:head [set-alloc-hint object_size 1048576 write_size 1048576,write 183513~13095] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.149965+0000", 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.185979756, 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029471900000000001, 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 505 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.149965+0000", 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.149965+0000", 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.149968+0000", 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0809999999999998e-06 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.150010+0000", 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1637999999999997e-05 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.150047+0000", 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6752000000000003e-05 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.150062+0000", 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4684e-05 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.150133+0000", 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1679000000000003e-05 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.150314+0000", 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018030699999999999 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.150497+0000", 2024-09-16T12:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000183022 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.151775+0000", 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001278398 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.152825+0000", 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00104958 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.152874+0000", 2024-09-16T12:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9546000000000003e-05 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.152913+0000", 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8503e-05 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:511 4.0 4:05d10a49:::rbd_data.11b59d02cc38.0000000000000044:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.178280+0000", 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15766561600000001, 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032036959999999998, 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 511 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178280+0000", 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178280+0000", 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178283+0000", 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4340000000000001e-06 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178299+0000", 2024-09-16T12:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5736999999999999e-05 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178353+0000", 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4437000000000001e-05 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178373+0000", 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0177e-05 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178456+0000", 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2798000000000002e-05 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178565+0000", 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108708 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178745+0000", 2024-09-16T12:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018054599999999999 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179984+0000", 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012383019999999999 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.181441+0000", 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014569629999999999 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.181464+0000", 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3796999999999999e-05 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.181483+0000", 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8797e-05 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:514 4.19 4:9af48b0e:::rbd_data.11b59d02cc38.000000000000008d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.178443+0000", 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15750239399999999, 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028705850000000002, 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 514 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178443+0000", 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178443+0000", 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178445+0000", 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8300000000000001e-06 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178460+0000", 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5170999999999999e-05 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178483+0000", 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3057e-05 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178495+0000", 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2515e-05 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178672+0000", 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017702500000000001 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178955+0000", 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028311100000000001 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179359+0000", 2024-09-16T12:59:17.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040325700000000002 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.180336+0000", 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097777299999999992 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.181250+0000", 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091393799999999999 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.181288+0000", 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000001e-05 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.181313+0000", 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4998e-05 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:515 4.1a e37/31 4:5f743d4b:::rbd_data.11b59d02cc38.00000000000000a4:head v 37'11, mlcod=36'9)", 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.179410+0000", 2024-09-16T12:59:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.156535379, 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015121119999999999, 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179410+0000", 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179410+0000", 2024-09-16T12:59:17.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179411+0000", 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8899999999999998e-07 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179417+0000", 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.652e-06 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179426+0000", 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0690000000000008e-06 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179433+0000", 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3040000000000001e-06 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179772+0000", 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000339692 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179794+0000", 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1759999999999998e-05 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.180890+0000", 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010955120000000001 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.180922+0000", 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2134000000000001e-05 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:522 4.1a e37/31 4:5da49aef:::rbd_data.11b59d02cc38.0000000000000001:head v 37'12, mlcod=37'10)", 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.193243+0000", 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.14270255500000001, 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080434600000000005, 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.193243+0000", 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.193243+0000", 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.193244+0000", 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.48e-06 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.193255+0000", 2024-09-16T12:59:17.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1154e-05 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.193283+0000", 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7321000000000002e-05 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.193325+0000", 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2573e-05 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.193391+0000", 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6115e-05 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.193410+0000", 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9038999999999998e-05 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.194004+0000", 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059378199999999995 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.194047+0000", 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2882e-05 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:544 4.17 4:efdedeac:::rbd_data.11b59d02cc38.000000000000004d:head [set-alloc-hint object_size 1048576 write_size 1048576,write 327680~28801] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.237694+0000", 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.098251322000000002, 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033377670000000002, 2024-09-16T12:59:17.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 544 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237694+0000", 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237694+0000", 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237698+0000", 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7629999999999998e-06 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237757+0000", 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9382999999999997e-05 2024-09-16T12:59:17.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237796+0000", 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9038999999999997e-05 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237810+0000", 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4347e-05 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237941+0000", 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013115699999999999 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.238194+0000", 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000252482 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.238462+0000", 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026800100000000002 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.240549+0000", 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020871309999999999 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.240820+0000", 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027090000000000003 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.240914+0000", 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4087999999999998e-05 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.241032+0000", 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117476 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:557 4.17 4:eab06920:::rbd_data.11b59d02cc38.0000000000000075:head [set-alloc-hint object_size 1048576 write_size 1048576,write 720896~5966] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.265124+0000", 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.070821084000000006, 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003336604, 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 557 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265124+0000", 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265124+0000", 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265127+0000", 2024-09-16T12:59:17.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.469e-06 2024-09-16T12:59:17.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265144+0000", 2024-09-16T12:59:17.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7050000000000001e-05 2024-09-16T12:59:17.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265175+0000", 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1779000000000002e-05 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265187+0000", 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1516e-05 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265303+0000", 2024-09-16T12:59:17.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116334 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265615+0000", 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031190800000000001 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265973+0000", 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035811600000000001 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.267565+0000", 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001591561 2024-09-16T12:59:17.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268334+0000", 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076968299999999996 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268407+0000", 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2152999999999998e-05 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268461+0000", 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4035e-05 2024-09-16T12:59:17.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:559 4.17 4:e9df2e13:::rbd_data.11b59d02cc38.0000000000000007:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.272189+0000", 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.063755949000000006, 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022074960000000002, 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 559 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.272189+0000", 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.272189+0000", 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.272191+0000", 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9580000000000001e-06 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.272201+0000", 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0145e-05 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.272229+0000", 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7256000000000001e-05 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.272249+0000", 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900999999999999e-05 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.272311+0000", 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1575999999999995e-05 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.272485+0000", 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000174394 2024-09-16T12:59:17.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:17.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.272675+0000", 2024-09-16T12:59:17.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189581 2024-09-16T12:59:17.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.274007+0000", 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00133226 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.274305+0000", 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029727099999999999 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.274356+0000", 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1496999999999998e-05 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.274397+0000", 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0657000000000001e-05 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:573 4.17 4:efdedeac:::rbd_data.11b59d02cc38.000000000000004d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.312227+0000", 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.023718507999999999, 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024514160000000001, 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 573 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312227+0000", 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312227+0000", 2024-09-16T12:59:17.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312230+0000", 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5839999999999999e-06 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312246+0000", 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5620999999999998e-05 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312294+0000", 2024-09-16T12:59:17.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8185e-05 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312308+0000", 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312395+0000", 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6897999999999994e-05 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312491+0000", 2024-09-16T12:59:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5475000000000003e-05 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312618+0000", 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012688399999999999 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313737+0000", 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011196859999999999 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314622+0000", 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088470900000000002 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314653+0000", 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1031999999999997e-05 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314678+0000", 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5058e-05 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:575 4.17 4:eab06920:::rbd_data.11b59d02cc38.0000000000000075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.023601158000000001, 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049089249999999997, 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 575 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:17.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312345+0000", 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.353e-06 2024-09-16T12:59:17.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312352+0000", 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7499999999999997e-06 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312370+0000", 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8210999999999999e-05 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312391+0000", 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0968e-05 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312965+0000", 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057344399999999999 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313134+0000", 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016979199999999999 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313322+0000", 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018742300000000001 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314160+0000", 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083821799999999995 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317190+0000", 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030301339999999999 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317219+0000", 2024-09-16T12:59:17.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8368999999999999e-05 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317253+0000", 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4263e-05 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:17.449 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:17.451 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:17.451 INFO:teuthology.orchestra.run.smithi042.stdout:187 trunc from 0x3474d61 to 0x64aea11 2024-09-16T12:59:17.451 INFO:teuthology.orchestra.run.smithi042.stdout:188 writesame 0x4a3ad2 thru 0x4a4617 data_size 0x25(0xb46 bytes) 2024-09-16T12:59:17.452 INFO:teuthology.orchestra.run.smithi042.stdout:191 writesame 0xd0ce6e3 thru 0xd0cf05c data_size 0x1(0x97a bytes) 2024-09-16T12:59:17.452 INFO:teuthology.orchestra.run.smithi042.stdout:193 write 0xe2ca04c thru 0xe2cecad (0x4c62 bytes) 2024-09-16T12:59:17.452 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:17.447+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:17.452 INFO:teuthology.orchestra.run.smithi042.stdout:194 trunc from 0xe2cecae to 0xe277930 2024-09-16T12:59:17.470 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.470 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.470 INFO:teuthology.orchestra.run.smithi042.stdout:199 read 0x2c76f1b thru 0x2c7a9e4 (0x3aca bytes) 2024-09-16T12:59:17.470 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.470 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.470 INFO:teuthology.orchestra.run.smithi042.stdout:202 read 0x46ab389 thru 0x46b0921 (0x5599 bytes) 2024-09-16T12:59:17.471 INFO:teuthology.orchestra.run.smithi042.stdout:203 read 0xaa3fdfd thru 0xaa4d18a (0xd38e bytes) 2024-09-16T12:59:17.471 INFO:teuthology.orchestra.run.smithi042.stdout:206 read 0xa69c9ff thru 0xa69de6a (0x146c bytes) 2024-09-16T12:59:17.471 INFO:teuthology.orchestra.run.smithi042.stdout:207 read 0x265d1c9 thru 0x26635f5 (0x642d bytes) 2024-09-16T12:59:17.471 INFO:teuthology.orchestra.run.smithi042.stdout:208 writesame 0x160ef9f thru 0x1615e35 data_size 0x1(0x6e97 bytes) 2024-09-16T12:59:17.503 INFO:teuthology.orchestra.run.smithi042.stdout:209 write 0x104df4e thru 0x105d92a (0xf9dd bytes) 2024-09-16T12:59:17.519 INFO:teuthology.orchestra.run.smithi042.stdout:210 trunc from 0xe277930 to 0xbc92fa3 2024-09-16T12:59:17.540 INFO:teuthology.orchestra.run.smithi042.stdout:211 punch from 0x3f54059 to 0x3f59460, (0x5407 bytes) 2024-09-16T12:59:17.540 INFO:teuthology.orchestra.run.smithi042.stdout:214 write 0x8ab244a thru 0x8ab442b (0x1fe2 bytes) 2024-09-16T12:59:17.552 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:17.547+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:17.554 INFO:teuthology.orchestra.run.smithi042.stdout:216 trunc from 0xbc92fa3 to 0x87571b8 2024-09-16T12:59:17.563 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:17.563 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:17.563 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:17.563 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:17.577 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:17.600 INFO:teuthology.orchestra.run.smithi042.stdout:218 writesame 0xeb82133 thru 0xeb840bc data_size 0xb(0x1f8a bytes) 2024-09-16T12:59:17.600 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.600 INFO:teuthology.orchestra.run.smithi042.stdout:222 write 0x139381a thru 0x1398a8b (0x5272 bytes) 2024-09-16T12:59:17.607 INFO:teuthology.orchestra.run.smithi042.stdout:223 read 0x523db4b thru 0x5243f7a (0x6430 bytes) 2024-09-16T12:59:17.607 INFO:teuthology.orchestra.run.smithi042.stdout:224 write 0x923d270 thru 0x9246798 (0x9529 bytes) 2024-09-16T12:59:17.615 INFO:teuthology.orchestra.run.smithi042.stdout:225 write 0x306ff51 thru 0x307a48f (0xa53f bytes) 2024-09-16T12:59:17.643 INFO:teuthology.orchestra.run.smithi042.stdout:229 trunc from 0xeb840bd to 0x663a184 2024-09-16T12:59:17.653 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:17.647+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:17.668 INFO:teuthology.orchestra.run.smithi042.stdout:230 read 0x4b8b782 thru 0x4b9b0e8 (0xf967 bytes) 2024-09-16T12:59:17.682 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:17.682 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:17.683 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:17.683 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:17.683 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:17.687 INFO:teuthology.orchestra.run.smithi042.stdout:232 write 0x8f4988c thru 0x8f58a02 (0xf177 bytes) 2024-09-16T12:59:17.693 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:17.695 INFO:teuthology.orchestra.run.smithi042.stdout:234 writesame 0x2c8482f thru 0x2c89b17 data_size 0x19(0x52e9 bytes) 2024-09-16T12:59:17.701 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.702 INFO:teuthology.orchestra.run.smithi042.stdout:237 writesame 0x5ab36b0 thru 0x5ab4f6e data_size 0x7(0x18bf bytes) 2024-09-16T12:59:17.708 INFO:teuthology.orchestra.run.smithi042.stdout:238 punch from 0x2d33418 to 0x2d34677, (0x125f bytes) 2024-09-16T12:59:17.708 INFO:teuthology.orchestra.run.smithi042.stdout:239 write 0x93c054 thru 0x941f32 (0x5edf bytes) 2024-09-16T12:59:17.716 INFO:teuthology.orchestra.run.smithi042.stdout:241 write 0x1e7e2ee thru 0x1e81b4f (0x3862 bytes) 2024-09-16T12:59:17.729 INFO:teuthology.orchestra.run.smithi042.stdout:242 trunc from 0x8f58a03 to 0x547fab0 2024-09-16T12:59:17.754 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:17.747+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout:243 write 0x578f6e2 thru 0x579abdc (0xb4fb bytes) 2024-09-16T12:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout:244 punch from 0x4692bb0 to 0x46a167b, (0xeacb bytes) 2024-09-16T12:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout:246 writesame 0x5f39476 thru 0x5f4243e data_size 0x1(0x8fc9 bytes) 2024-09-16T12:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout:249 read 0x51ec4f3 thru 0x51fb8ca (0xf3d8 bytes) 2024-09-16T12:59:17.796 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:17.796 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:17.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:17.796 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:17.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.796 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:7 4.1a 4:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T12:59:17.796 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:12.759313+0000", 2024-09-16T12:59:17.796 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0330577889999999, 2024-09-16T12:59:17.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074382090000000003, 2024-09-16T12:59:17.796 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 7 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759313+0000", 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759313+0000", 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759316+0000", 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.213e-06 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759331+0000", 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5082999999999999e-05 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759368+0000", 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6415999999999998e-05 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759384+0000", 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6039999999999999e-05 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759547+0000", 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163466 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759563+0000", 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5262999999999999e-05 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.764413+0000", 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048502859999999997 2024-09-16T12:59:17.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.764694+0000", 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028159099999999998 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.764933+0000", 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023883300000000001 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.766322+0000", 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013884500000000001 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.766664+0000", 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034210000000000002 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.766717+0000", 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2932000000000001e-05 2024-09-16T12:59:17.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.766751+0000", 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4536000000000003e-05 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:9 4.1a 4:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:13.777083+0000", 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0152882849999996, 2024-09-16T12:59:17.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045017440000000002, 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 9 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.777083+0000", 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.777083+0000", 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.777085+0000", 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9690000000000001e-06 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.777093+0000", 2024-09-16T12:59:17.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7620000000000001e-06 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.777114+0000", 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0335000000000001e-05 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.777125+0000", 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1576999999999999e-05 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.777884+0000", 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000759047 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.778149+0000", 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026498500000000003 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.778512+0000", 2024-09-16T12:59:17.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000362288 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.779989+0000", 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014772349999999999 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.781502+0000", 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001512829 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.781551+0000", 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8940999999999998e-05 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.781584+0000", 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3776000000000003e-05 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:12 4.c 4:35454ec4:::rbd_header.11b524d5182b:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:13.795162+0000", 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9972086820000001, 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027703070000000001, 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 12 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.795162+0000", 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.795162+0000", 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.795165+0000", 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3080000000000002e-06 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.795175+0000", 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0002e-05 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.795202+0000", 2024-09-16T12:59:17.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7277e-05 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.795216+0000", 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4321999999999999e-05 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.795360+0000", 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014354400000000001 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.795667+0000", 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030729599999999998 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.796259+0000", 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059236399999999995 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.797573+0000", 2024-09-16T12:59:17.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001313369 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.797749+0000", 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000176513 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.797855+0000", 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010603900000000001 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.797933+0000", 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7273000000000004e-05 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:379 4.13 4:cb104334:::rbd_object_map.11b524d5182b:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:16.911368+0000", 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.88100288599999999, 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028154270000000001, 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.816 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.911368+0000", 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.911368+0000", 2024-09-16T12:59:17.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.911370+0000", 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7999999999999999e-06 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.911379+0000", 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0040000000000005e-06 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.911406+0000", 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7163999999999999e-05 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.911434+0000", 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7645999999999999e-05 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.911610+0000", 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017617500000000001 2024-09-16T12:59:17.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.911869+0000", 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025899799999999998 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.912290+0000", 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042093300000000002 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.913360+0000", 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001070451 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.913951+0000", 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059032400000000004 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.914091+0000", 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014014 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.914183+0000", 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2792000000000002e-05 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:428 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 956960~24840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:16.972359+0000", 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.82001225, 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030633180000000002, 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.972359+0000", 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.972359+0000", 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.972361+0000", 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0449999999999999e-06 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.972409+0000", 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8606000000000002e-05 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.972437+0000", 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7942000000000001e-05 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.972448+0000", 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0688e-05 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.972545+0000", 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6626999999999995e-05 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.972742+0000", 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197011 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.972951+0000", 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020897299999999999 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.974385+0000", 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014341099999999999 2024-09-16T12:59:17.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.975272+0000", 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088691699999999998 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.975338+0000", 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5969000000000005e-05 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.975422+0000", 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4430000000000003e-05 2024-09-16T12:59:17.827 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:430 4.e 4:729a1ad2:::rbd_header.11b59d02cc38:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:16.996274+0000", 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.79609657300000003, 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028493210000000001, 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.828 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.996274+0000", 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.996274+0000", 2024-09-16T12:59:17.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.996277+0000", 2024-09-16T12:59:17.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.323e-06 2024-09-16T12:59:17.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.996288+0000", 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1697e-05 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.996321+0000", 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2966000000000003e-05 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.996335+0000", 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3283e-05 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.996439+0000", 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010425399999999999 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.996589+0000", 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149763 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.996908+0000", 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031939700000000002 2024-09-16T12:59:17.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.998062+0000", 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001154111 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.998969+0000", 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00090719100000000001 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.999063+0000", 2024-09-16T12:59:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3424000000000005e-05 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.999124+0000", 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0912000000000002e-05 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:481 4.1a 4:5f743d4b:::rbd_data.11b59d02cc38.00000000000000a4:head [set-alloc-hint object_size 1048576 write_size 1048576,write 834800~17168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.100277+0000", 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.69209414800000002, 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031740879999999998, 2024-09-16T12:59:17.834 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 481 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100277+0000", 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100277+0000", 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100279+0000", 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0169999999999999e-06 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100304+0000", 2024-09-16T12:59:17.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5304999999999999e-05 2024-09-16T12:59:17.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100331+0000", 2024-09-16T12:59:17.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6554e-05 2024-09-16T12:59:17.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100341+0000", 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.081e-05 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100446+0000", 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104644 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100791+0000", 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034490900000000001 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.101052+0000", 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026069100000000002 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.102983+0000", 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019317729999999999 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.103067+0000", 2024-09-16T12:59:17.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3800000000000004e-05 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.103367+0000", 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029928300000000001 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.103451+0000", 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4301999999999993e-05 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:482 4.e 4:71fee990:::rbd_data.11b59d02cc38.00000000000000a5:head [set-alloc-hint object_size 1048576 write_size 1048576,write 786432~48108] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.100862+0000", 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.69150844499999997, 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033455500000000001, 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 482 2024-09-16T12:59:17.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100862+0000", 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100862+0000", 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100864+0000", 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7230000000000001e-06 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100937+0000", 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2421000000000003e-05 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100961+0000", 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4334000000000001e-05 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.100970+0000", 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4420000000000004e-06 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.101059+0000", 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8123999999999999e-05 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.101334+0000", 2024-09-16T12:59:17.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027508300000000002 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.101692+0000", 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035877499999999998 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.103765+0000", 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020728890000000001 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.104053+0000", 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028738 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.104144+0000", 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1147e-05 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.104208+0000", 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4232000000000004e-05 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:506 4.1 4:81b004bd:::rbd_data.11b59d02cc38.0000000000000045:head [set-alloc-hint object_size 1048576 write_size 1048576,write 131072~52258] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.153564+0000", 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.63880658300000004, 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032149240000000001, 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.847 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 506 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.153564+0000", 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.153564+0000", 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.153566+0000", 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0030000000000001e-06 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.153668+0000", 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101315 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.153706+0000", 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8463000000000001e-05 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.153713+0000", 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9709999999999998e-06 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.153849+0000", 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013638600000000001 2024-09-16T12:59:17.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.154149+0000", 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000299351 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.154421+0000", 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027173099999999999 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.156012+0000", 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015910010000000001 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.156660+0000", 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00064882000000000004 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.156731+0000", 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0588999999999998e-05 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.156779+0000", 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8294000000000002e-05 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:510 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.180926+0000", 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.61144442200000004, 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054779440000000002, 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.853 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 510 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180926+0000", 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180926+0000", 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180929+0000", 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4849999999999999e-06 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180942+0000", 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2886e-05 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180975+0000", 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2907999999999997e-05 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180991+0000", 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6507999999999998e-05 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181129+0000", 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000137758 2024-09-16T12:59:17.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181317+0000", 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000188359 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181501+0000", 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018385299999999999 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.185589+0000", 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040873439999999997 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186180+0000", 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059100999999999999 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186326+0000", 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014665699999999999 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186404+0000", 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8176000000000005e-05 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:512 4.1 4:81b004bd:::rbd_data.11b59d02cc38.0000000000000045:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.181053+0000", 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.61131809100000001, 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055154790000000002, 2024-09-16T12:59:17.859 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181053+0000", 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181053+0000", 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181054+0000", 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5999999999999999e-06 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181063+0000", 2024-09-16T12:59:17.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0000000000000002e-06 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181086+0000", 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3057e-05 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181098+0000", 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1634000000000001e-05 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181210+0000", 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111687 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181411+0000", 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020129300000000001 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181715+0000", 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030415299999999999 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186212+0000", 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044970360000000003 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186308+0000", 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5587999999999994e-05 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186458+0000", 2024-09-16T12:59:17.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015040600000000001 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186568+0000", 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110025 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:515 4.1a 4:5f743d4b:::rbd_data.11b59d02cc38.00000000000000a4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.865 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.181208+0000", 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.61116332799999995, 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054488139999999997, 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 515 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181208+0000", 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181208+0000", 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181209+0000", 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.902e-06 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181218+0000", 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3119999999999994e-06 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181239+0000", 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0811000000000001e-05 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181250+0000", 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1824999999999999e-05 2024-09-16T12:59:17.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181375+0000", 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000124986 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181597+0000", 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022121 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181909+0000", 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031264999999999998 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183971+0000", 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002061401 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186444+0000", 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024731229999999998 2024-09-16T12:59:17.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186562+0000", 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118546 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186656+0000", 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4048000000000005e-05 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:516 4.e 4:71fee990:::rbd_data.11b59d02cc38.00000000000000a5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.181283+0000", 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.61108817299999996, 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055893929999999998, 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 516 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181283+0000", 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181283+0000", 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181284+0000", 2024-09-16T12:59:17.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3859999999999999e-06 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181290+0000", 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0079999999999999e-06 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181307+0000", 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6728999999999999e-05 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181324+0000", 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6830999999999999e-05 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.182256+0000", 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093190299999999998 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.182486+0000", 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023028600000000001 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183206+0000", 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000720431 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186616+0000", 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034098079999999998 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186757+0000", 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014120199999999999 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186822+0000", 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4793000000000001e-05 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186872+0000", 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0015999999999997e-05 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:590 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.45310146499999998, 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034258392999999998, 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 590 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339271+0000", 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339281+0000", 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6670000000000008e-06 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339307+0000", 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5907000000000001e-05 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339317+0000", 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0356999999999999e-05 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339412+0000", 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4248999999999998e-05 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339533+0000", 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012184900000000001 2024-09-16T12:59:17.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.347489+0000", 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079559599999999998 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373423+0000", 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025933971 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373435+0000", 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1924e-05 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373492+0000", 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6842999999999998e-05 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373528+0000", 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-16T12:59:17.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:590 4.e e37/31 ondisk, result = 0)", 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.366505+0000", 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.42586640399999998, 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069213820000000002, 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.366505+0000", 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.366505+0000", 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.366506+0000", 2024-09-16T12:59:17.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2470000000000001e-06 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.366513+0000", 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2490000000000004e-06 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.366533+0000", 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9882e-05 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.366542+0000", 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7690000000000002e-06 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373405+0000", 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068630119999999999 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373415+0000", 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9259999999999995e-06 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373426+0000", 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1297e-05 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:622 4.d 4:b218b9b2:::rbd_data.11b59d02cc38.00000000000000e2:head [set-alloc-hint object_size 1048576 write_size 1048576,write 434252~19554] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.447526+0000", 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.34484516799999998, 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035828190000000001, 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.888 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 622 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.447526+0000", 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.447526+0000", 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.447528+0000", 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.447560+0000", 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2206000000000003e-05 2024-09-16T12:59:17.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.447591+0000", 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1180000000000003e-05 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.447603+0000", 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2002e-05 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.447718+0000", 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114882 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.447953+0000", 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000234921 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.448207+0000", 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025371999999999998 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.449914+0000", 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017071359999999999 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.450842+0000", 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092833800000000001 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.450993+0000", 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150674 2024-09-16T12:59:17.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.451109+0000", 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011543 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:623 4.d 4:b218b9b2:::rbd_data.11b59d02cc38.00000000000000e2:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.451649+0000", 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.34072205100000003, 2024-09-16T12:59:17.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00358107, 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 623 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.451649+0000", 2024-09-16T12:59:17.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.451649+0000", 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.451651+0000", 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3e-06 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.451663+0000", 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1574e-05 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.451694+0000", 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1495999999999998e-05 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.451706+0000", 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2217e-05 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.451813+0000", 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106467 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.452056+0000", 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000242721 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:17.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.452465+0000", 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040926500000000002 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.453928+0000", 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001462603 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.455069+0000", 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011410669999999999 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.455162+0000", 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3270999999999995e-05 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.455230+0000", 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8089000000000005e-05 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:640 4.18 4:1b76a142:::rbd_data.11b59d02cc38.0000000000000011:head [set-alloc-hint object_size 1048576 write_size 1048576,write 131072~55595] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.508916+0000", 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.28345493399999999, 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029317789999999998, 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 640 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508916+0000", 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508916+0000", 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508919+0000", 2024-09-16T12:59:17.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9890000000000001e-06 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509013+0000", 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4303999999999997e-05 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509039+0000", 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6088000000000002e-05 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509049+0000", 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9229999999999997e-06 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509161+0000", 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111424 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509353+0000", 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019252000000000001 2024-09-16T12:59:17.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509535+0000", 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018145699999999999 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.510664+0000", 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001129681 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.511730+0000", 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010655930000000001 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.511792+0000", 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1719999999999999e-05 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.511848+0000", 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6079999999999998e-05 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.906 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:690 4.e 4:729a1ad2:::rbd_header.11b59d02cc38:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.653494+0000", 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.138876477, 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028415960000000001, 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.907 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 690 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.653494+0000", 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.653494+0000", 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.653498+0000", 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1939999999999998e-06 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.653511+0000", 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3026e-05 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.653548+0000", 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7188999999999999e-05 2024-09-16T12:59:17.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.653567+0000", 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9202999999999999e-05 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.653689+0000", 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012233799999999999 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.653836+0000", 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146763 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.654158+0000", 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000321627 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.655186+0000", 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001028155 2024-09-16T12:59:17.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.656041+0000", 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085537000000000005 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.656200+0000", 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015864399999999999 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.656336+0000", 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013608700000000001 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:713 4.d 4:b5c6435a:::rbd_data.11b59d02cc38.000000000000002c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.698015+0000", 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.094355835999999998, 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029351730000000001, 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:17.913 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 713 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.698015+0000", 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:17.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.698015+0000", 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.698018+0000", 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7460000000000001e-06 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.698035+0000", 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6719e-05 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.698066+0000", 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1044999999999998e-05 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.698078+0000", 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.258e-05 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.698155+0000", 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6542e-05 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.698314+0000", 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015883499999999999 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.698547+0000", 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000233367 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.699856+0000", 2024-09-16T12:59:17.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013090860000000001 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.700893+0000", 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010374360000000001 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.700927+0000", 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3145000000000002e-05 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:17.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.700950+0000", 2024-09-16T12:59:17.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3672e-05 2024-09-16T12:59:17.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.919 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:17.919 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:17.919 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:17.920 INFO:teuthology.orchestra.run.smithi042.stdout:251 write 0x6385efd thru 0x639103f (0xb143 bytes) 2024-09-16T12:59:17.920 INFO:teuthology.orchestra.run.smithi042.stdout:252 read 0x1d97f55 thru 0x1da3b63 (0xbc0f bytes) 2024-09-16T12:59:17.921 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:17.922 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:17.915+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:17.922 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T12:59:17.923 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T12:59:17.923 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T12:59:17.924 INFO:teuthology.orchestra.run.smithi042.stdout:253 write 0xb54efd7 thru 0xb54fbf8 (0xc22 bytes) 2024-09-16T12:59:17.924 INFO:teuthology.orchestra.run.smithi042.stdout:254 writesame 0xddcd5e2 thru 0xddcf8a6 data_size 0x2b(0x22c5 bytes) 2024-09-16T12:59:17.924 INFO:teuthology.orchestra.run.smithi042.stdout:255 trunc from 0xddcf8a7 to 0xb9fc7b5 2024-09-16T12:59:17.924 INFO:teuthology.orchestra.run.smithi042.stdout:257 read 0x5096866 thru 0x50983fa (0x1b95 bytes) 2024-09-16T12:59:17.924 INFO:teuthology.orchestra.run.smithi042.stdout:259 write 0xdccc567 thru 0xdccfdd9 (0x3873 bytes) 2024-09-16T12:59:17.924 INFO:teuthology.orchestra.run.smithi042.stdout:261 read 0x64d4e0f thru 0x64dc9f6 (0x7be8 bytes) 2024-09-16T12:59:17.924 INFO:teuthology.orchestra.run.smithi042.stdout:263 read 0x493f0c8 thru 0x494cbdc (0xdb15 bytes) 2024-09-16T12:59:17.924 INFO:teuthology.orchestra.run.smithi042.stdout:265 read 0x75de6ef thru 0x75e542f (0x6d41 bytes) 2024-09-16T12:59:17.924 INFO:teuthology.orchestra.run.smithi042.stdout:267 write 0xe29f23e thru 0xe2a66ce (0x7491 bytes) 2024-09-16T12:59:17.928 INFO:teuthology.orchestra.run.smithi042.stdout:268 write 0x5a55fe9 thru 0x5a61bf3 (0xbc0b bytes) 2024-09-16T12:59:17.941 INFO:teuthology.orchestra.run.smithi042.stdout:269 trunc from 0xe2a66cf to 0xcd0a965 2024-09-16T12:59:17.962 INFO:teuthology.orchestra.run.smithi042.stdout:270 write 0x76abfd7 thru 0x76b2ba6 (0x6bd0 bytes) 2024-09-16T12:59:17.970 INFO:teuthology.orchestra.run.smithi042.stdout:272 read 0x134e4f1 thru 0x135896e (0xa47e bytes) 2024-09-16T12:59:17.975 INFO:teuthology.orchestra.run.smithi042.stdout:273 write 0x17c3042 thru 0x17c94c8 (0x6487 bytes) 2024-09-16T12:59:17.979 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:17.980 INFO:teuthology.orchestra.run.smithi042.stdout:275 trunc from 0xcd0a965 to 0xd898c65 2024-09-16T12:59:17.992 INFO:teuthology.orchestra.run.smithi042.stdout:276 write 0x7dd8147 thru 0x7dd8521 (0x3db bytes) 2024-09-16T12:59:17.998 INFO:teuthology.orchestra.run.smithi042.stdout:277 write 0x29e3698 thru 0x29e5af9 (0x2462 bytes) 2024-09-16T12:59:18.005 INFO:teuthology.orchestra.run.smithi042.stdout:278 read 0x3d56ea4 thru 0x3d632ad (0xc40a bytes) 2024-09-16T12:59:18.021 INFO:teuthology.orchestra.run.smithi042.stdout:279 trunc from 0xd898c65 to 0x6327324 2024-09-16T12:59:18.021 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:18.015+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:18.042 INFO:teuthology.orchestra.run.smithi042.stdout:280 read 0x15e064f thru 0x15ef9ff (0xf3b1 bytes) 2024-09-16T12:59:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:281 write 0x8ac5121 thru 0x8ac989b (0x477b bytes) 2024-09-16T12:59:18.064 INFO:teuthology.orchestra.run.smithi042.stdout:282 read 0x608421d thru 0x608b754 (0x7538 bytes) 2024-09-16T12:59:18.064 INFO:teuthology.orchestra.run.smithi042.stdout:284 read 0x8a14fb8 thru 0x8a1a0f5 (0x513e bytes) 2024-09-16T12:59:18.075 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:18.075 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:18.075 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:18.075 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:18.081 INFO:teuthology.orchestra.run.smithi042.stdout:285 write 0xc1005b3 thru 0xc10c917 (0xc365 bytes) 2024-09-16T12:59:18.086 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:18.088 INFO:teuthology.orchestra.run.smithi042.stdout:286 write 0x5b78a3c thru 0x5b7bc17 (0x31dc bytes) 2024-09-16T12:59:18.098 INFO:teuthology.orchestra.run.smithi042.stdout:287 trunc from 0xc10c918 to 0x922f152 2024-09-16T12:59:18.117 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:18.117 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:18.117 INFO:teuthology.orchestra.run.smithi042.stdout:291 write 0x1644d27 thru 0x164e4be (0x9798 bytes) 2024-09-16T12:59:18.120 INFO:teuthology.orchestra.run.smithi042.stdout:292 read 0x1e2b3c9 thru 0x1e3ace3 (0xf91b bytes) 2024-09-16T12:59:18.121 INFO:teuthology.orchestra.run.smithi042.stdout:293 punch from 0x2859bf0 to 0x2860208, (0x6618 bytes) 2024-09-16T12:59:18.122 INFO:teuthology.orchestra.run.smithi042.stdout:294 punch from 0x5688d6e to 0x5692188, (0x941a bytes) 2024-09-16T12:59:18.122 INFO:teuthology.orchestra.run.smithi042.stdout:295 read 0xefbaba thru 0xf016ad (0x5bf4 bytes) 2024-09-16T12:59:18.122 INFO:teuthology.orchestra.run.smithi042.stdout:296 read 0x4d182b3 thru 0x4d1a4dc (0x222a bytes) 2024-09-16T12:59:18.122 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:18.122 INFO:teuthology.orchestra.run.smithi042.stdout:299 writesame 0x85638f8 thru 0x8564241 data_size 0x2(0x94a bytes) 2024-09-16T12:59:18.123 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:18.119+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:18.128 INFO:teuthology.orchestra.run.smithi042.stdout:300 read 0x3007fd2 thru 0x300c470 (0x449f bytes) 2024-09-16T12:59:18.134 INFO:teuthology.orchestra.run.smithi042.stdout:301 trunc from 0x922f152 to 0x6e88b9f 2024-09-16T12:59:18.155 INFO:teuthology.orchestra.run.smithi042.stdout:303 write 0x21369a1 thru 0x213c551 (0x5bb1 bytes) 2024-09-16T12:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:304 punch from 0x493cbbb to 0x494a04e, (0xd493 bytes) 2024-09-16T12:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:306 read 0x3113524 thru 0x3114b8e (0x166b bytes) 2024-09-16T12:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:308 read 0x5635bb0 thru 0x563b879 (0x5cca bytes) 2024-09-16T12:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout:309 write 0x9dc55cb thru 0x9dca7b3 (0x51e9 bytes) 2024-09-16T12:59:18.187 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:18.187 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:18.187 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:18.187 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:18.187 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout:310 writesame 0x8f46b2b thru 0x8f4b313 data_size 0x1(0x47e9 bytes) 2024-09-16T12:59:18.196 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout:311 write 0x7ea6d1 thru 0x7f8d9a (0xe6ca bytes) 2024-09-16T12:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout:314 write 0x502d12d thru 0x50311ce (0x40a2 bytes) 2024-09-16T12:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout:316 punch from 0x25655e6 to 0x25663d7, (0xdf1 bytes) 2024-09-16T12:59:18.223 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:18.219+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout:317 trunc from 0x9dca7b4 to 0xc67bd15 2024-09-16T12:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout:318 write 0xc30acf2 thru 0xc30e646 (0x3955 bytes) 2024-09-16T12:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout:319 read 0x5aebb42 thru 0x5af938e (0xd84d bytes) 2024-09-16T12:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout:320 read 0x3956cec thru 0x395f54e (0x8863 bytes) 2024-09-16T12:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout:321 writesame 0x1f48427 thru 0x1f51bed data_size 0x13(0x97c7 bytes) 2024-09-16T12:59:18.248 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T12:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout:322 write 0x35f4ee9 thru 0x35fe563 (0x967b bytes) 2024-09-16T12:59:18.255 INFO:teuthology.orchestra.run.smithi042.stdout:323 read 0x4f65c88 thru 0x4f700de (0xa457 bytes) 2024-09-16T12:59:18.255 INFO:teuthology.orchestra.run.smithi042.stdout:324 write 0x3531dc5 thru 0x353a75a (0x8996 bytes) 2024-09-16T12:59:18.260 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-16T12:59:17.800745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":100,"num_read_kb":81,"num_write":109,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411016,"kb_used_data":3024,"kb_used_omap":13,"kb_used_meta":2407858,"kb_avail":752563704,"statfs":{"total":773094113280,"available":770625232896,"internally_reserved":0,"allocated":3096576,"data_stored":2389958,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2465647296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027460"},"pg_stats":[{"pgid":"4.1f","version":"34'1","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.804599+0000","last_change":"2024-09-16T12:59:13.789786+0000","last_active":"2024-09-16T12:59:13.804599+0000","last_peered":"2024-09-16T12:59:13.804599+0000","last_clean":"2024-09-16T12:59:13.804599+0000","last_became_active":"2024-09-16T12:59:11.766798+0000","last_became_peered":"2024-09-16T12:59:11.766798+0000","last_unstale":"2024-09-16T12:59:13.804599+0000","last_undegraded":"2024-09-16T12:59:13.804599+0000","last_fullsized":"2024-09-16T12:59:13.804599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:22.588888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.030274+0000","last_change":"2024-09-16T12:59:14.030480+0000","last_active":"2024-09-16T12:59:14.030274+0000","last_peered":"2024-09-16T12:59:14.030274+0000","last_clean":"2024-09-16T12:59:14.030274+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T12:59:14.030274+0000","last_undegraded":"2024-09-16T12:59:14.030274+0000","last_fullsized":"2024-09-16T12:59:14.030274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049602200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.769948+0000","last_change":"2024-09-16T12:59:13.770162+0000","last_active":"2024-09-16T12:59:13.769948+0000","last_peered":"2024-09-16T12:59:13.769948+0000","last_clean":"2024-09-16T12:59:13.769948+0000","last_became_active":"2024-09-16T12:59:11.761823+0000","last_became_peered":"2024-09-16T12:59:11.761823+0000","last_unstale":"2024-09-16T12:59:13.769948+0000","last_undegraded":"2024-09-16T12:59:13.769948+0000","last_fullsized":"2024-09-16T12:59:13.769948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:33.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012332339999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1c","version":"34'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.782901+0000","last_change":"2024-09-16T12:59:13.770236+0000","last_active":"2024-09-16T12:59:13.782901+0000","last_peered":"2024-09-16T12:59:13.782901+0000","last_clean":"2024-09-16T12:59:13.782901+0000","last_became_active":"2024-09-16T12:59:11.766604+0000","last_became_peered":"2024-09-16T12:59:11.766604+0000","last_unstale":"2024-09-16T12:59:13.782901+0000","last_undegraded":"2024-09-16T12:59:13.782901+0000","last_fullsized":"2024-09-16T12:59:13.782901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:01.891108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015138090000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.779899+0000","last_change":"2024-09-16T12:59:13.779899+0000","last_active":"2024-09-16T12:59:13.779899+0000","last_peered":"2024-09-16T12:59:13.779899+0000","last_clean":"2024-09-16T12:59:13.779899+0000","last_became_active":"2024-09-16T12:59:11.759998+0000","last_became_peered":"2024-09-16T12:59:11.759998+0000","last_unstale":"2024-09-16T12:59:13.779899+0000","last_undegraded":"2024-09-16T12:59:13.779899+0000","last_fullsized":"2024-09-16T12:59:13.779899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:49:22.613940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002573915,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1a","version":"34'2","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.814406+0000","last_change":"2024-09-16T12:59:13.779650+0000","last_active":"2024-09-16T12:59:13.814406+0000","last_peered":"2024-09-16T12:59:13.814406+0000","last_clean":"2024-09-16T12:59:13.814406+0000","last_became_active":"2024-09-16T12:59:11.759751+0000","last_became_peered":"2024-09-16T12:59:11.759751+0000","last_unstale":"2024-09-16T12:59:13.814406+0000","last_undegraded":"2024-09-16T12:59:13.814406+0000","last_fullsized":"2024-09-16T12:59:13.814406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:12.049622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002565964,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.770227+0000","last_change":"2024-09-16T12:59:13.770391+0000","last_active":"2024-09-16T12:59:13.770227+0000","last_peered":"2024-09-16T12:59:13.770227+0000","last_clean":"2024-09-16T12:59:13.770227+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T12:59:13.770227+0000","last_undegraded":"2024-09-16T12:59:13.770227+0000","last_fullsized":"2024-09-16T12:59:13.770227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013782849999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.777900+0000","last_change":"2024-09-16T12:59:13.777900+0000","last_active":"2024-09-16T12:59:13.777900+0000","last_peered":"2024-09-16T12:59:13.777900+0000","last_clean":"2024-09-16T12:59:13.777900+0000","last_became_active":"2024-09-16T12:59:11.767308+0000","last_became_peered":"2024-09-16T12:59:11.767308+0000","last_unstale":"2024-09-16T12:59:13.777900+0000","last_undegraded":"2024-09-16T12:59:13.777900+0000","last_fullsized":"2024-09-16T12:59:13.777900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:40.593369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093274999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.770445+0000","last_change":"2024-09-16T12:59:13.770695+0000","last_active":"2024-09-16T12:59:13.770445+0000","last_peered":"2024-09-16T12:59:13.770445+0000","last_clean":"2024-09-16T12:59:13.770445+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T12:59:13.770445+0000","last_undegraded":"2024-09-16T12:59:13.770445+0000","last_fullsized":"2024-09-16T12:59:13.770445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001481132,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.030536+0000","last_change":"2024-09-16T12:59:14.030762+0000","last_active":"2024-09-16T12:59:14.030536+0000","last_peered":"2024-09-16T12:59:14.030536+0000","last_clean":"2024-09-16T12:59:14.030536+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T12:59:14.030536+0000","last_undegraded":"2024-09-16T12:59:14.030536+0000","last_fullsized":"2024-09-16T12:59:14.030536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685533,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.796139+0000","last_change":"2024-09-16T12:59:13.796422+0000","last_active":"2024-09-16T12:59:13.796139+0000","last_peered":"2024-09-16T12:59:13.796139+0000","last_clean":"2024-09-16T12:59:13.796139+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T12:59:13.796139+0000","last_undegraded":"2024-09-16T12:59:13.796139+0000","last_fullsized":"2024-09-16T12:59:13.796139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055065799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.770518+0000","last_change":"2024-09-16T12:59:13.770810+0000","last_active":"2024-09-16T12:59:13.770518+0000","last_peered":"2024-09-16T12:59:13.770518+0000","last_clean":"2024-09-16T12:59:13.770518+0000","last_became_active":"2024-09-16T12:59:11.767626+0000","last_became_peered":"2024-09-16T12:59:11.767626+0000","last_unstale":"2024-09-16T12:59:13.770518+0000","last_undegraded":"2024-09-16T12:59:13.770518+0000","last_fullsized":"2024-09-16T12:59:13.770518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:13.669310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013482349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.13","version":"34'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.800199+0000","last_change":"2024-09-16T12:59:13.780086+0000","last_active":"2024-09-16T12:59:13.800199+0000","last_peered":"2024-09-16T12:59:13.800199+0000","last_clean":"2024-09-16T12:59:13.800199+0000","last_became_active":"2024-09-16T12:59:11.770883+0000","last_became_peered":"2024-09-16T12:59:11.770883+0000","last_unstale":"2024-09-16T12:59:13.800199+0000","last_undegraded":"2024-09-16T12:59:13.800199+0000","last_fullsized":"2024-09-16T12:59:13.800199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:22.020502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024518000000000001,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.783323+0000","last_change":"2024-09-16T12:59:13.783659+0000","last_active":"2024-09-16T12:59:13.783323+0000","last_peered":"2024-09-16T12:59:13.783323+0000","last_clean":"2024-09-16T12:59:13.783323+0000","last_became_active":"2024-09-16T12:59:11.768690+0000","last_became_peered":"2024-09-16T12:59:11.768690+0000","last_unstale":"2024-09-16T12:59:13.783323+0000","last_undegraded":"2024-09-16T12:59:13.783323+0000","last_fullsized":"2024-09-16T12:59:13.783323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:00.837257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061900799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.769884+0000","last_change":"2024-09-16T12:59:13.770311+0000","last_active":"2024-09-16T12:59:13.769884+0000","last_peered":"2024-09-16T12:59:13.769884+0000","last_clean":"2024-09-16T12:59:13.769884+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T12:59:13.769884+0000","last_undegraded":"2024-09-16T12:59:13.769884+0000","last_fullsized":"2024-09-16T12:59:13.769884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413508,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.769309+0000","last_change":"2024-09-16T12:59:13.769620+0000","last_active":"2024-09-16T12:59:13.769309+0000","last_peered":"2024-09-16T12:59:13.769309+0000","last_clean":"2024-09-16T12:59:13.769309+0000","last_became_active":"2024-09-16T12:59:11.763566+0000","last_became_peered":"2024-09-16T12:59:11.763566+0000","last_unstale":"2024-09-16T12:59:13.769309+0000","last_undegraded":"2024-09-16T12:59:13.769309+0000","last_fullsized":"2024-09-16T12:59:13.769309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:16.800665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047952,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.119623+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T12:59:13.119623+0000","last_peered":"2024-09-16T12:59:13.119623+0000","last_clean":"2024-09-16T12:59:13.119623+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T12:59:13.119623+0000","last_undegraded":"2024-09-16T12:59:13.119623+0000","last_fullsized":"2024-09-16T12:59:13.119623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.777645+0000","last_change":"2024-09-16T12:59:13.777645+0000","last_active":"2024-09-16T12:59:13.777645+0000","last_peered":"2024-09-16T12:59:13.777645+0000","last_clean":"2024-09-16T12:59:13.777645+0000","last_became_active":"2024-09-16T12:59:11.770325+0000","last_became_peered":"2024-09-16T12:59:11.770325+0000","last_unstale":"2024-09-16T12:59:13.777645+0000","last_undegraded":"2024-09-16T12:59:13.777645+0000","last_fullsized":"2024-09-16T12:59:13.777645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:06:31.319709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069048899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.781099+0000","last_change":"2024-09-16T12:58:55.653750+0000","last_active":"2024-09-16T12:59:12.781099+0000","last_peered":"2024-09-16T12:59:12.781099+0000","last_clean":"2024-09-16T12:59:12.781099+0000","last_became_active":"2024-09-16T12:58:53.634053+0000","last_became_peered":"2024-09-16T12:58:53.634053+0000","last_unstale":"2024-09-16T12:59:12.781099+0000","last_undegraded":"2024-09-16T12:59:12.781099+0000","last_fullsized":"2024-09-16T12:59:12.781099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:50.402394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065616299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.769355+0000","last_change":"2024-09-16T12:59:13.769781+0000","last_active":"2024-09-16T12:59:13.769355+0000","last_peered":"2024-09-16T12:59:13.769355+0000","last_clean":"2024-09-16T12:59:13.769355+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T12:59:13.769355+0000","last_undegraded":"2024-09-16T12:59:13.769355+0000","last_fullsized":"2024-09-16T12:59:13.769355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010481990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.717846+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T12:59:13.717846+0000","last_peered":"2024-09-16T12:59:13.717846+0000","last_clean":"2024-09-16T12:59:13.717846+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T12:59:13.717846+0000","last_undegraded":"2024-09-16T12:59:13.717846+0000","last_fullsized":"2024-09-16T12:59:13.717846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.030580+0000","last_change":"2024-09-16T12:59:14.030767+0000","last_active":"2024-09-16T12:59:14.030580+0000","last_peered":"2024-09-16T12:59:14.030580+0000","last_clean":"2024-09-16T12:59:14.030580+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T12:59:14.030580+0000","last_undegraded":"2024-09-16T12:59:14.030580+0000","last_fullsized":"2024-09-16T12:59:14.030580+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000670212,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.780996+0000","last_change":"2024-09-16T12:58:55.653753+0000","last_active":"2024-09-16T12:59:12.780996+0000","last_peered":"2024-09-16T12:59:12.780996+0000","last_clean":"2024-09-16T12:59:12.780996+0000","last_became_active":"2024-09-16T12:58:53.630727+0000","last_became_peered":"2024-09-16T12:58:53.630727+0000","last_unstale":"2024-09-16T12:59:12.780996+0000","last_undegraded":"2024-09-16T12:59:12.780996+0000","last_fullsized":"2024-09-16T12:59:12.780996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:00.855732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061053599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.783361+0000","last_change":"2024-09-16T12:59:13.783668+0000","last_active":"2024-09-16T12:59:13.783361+0000","last_peered":"2024-09-16T12:59:13.783361+0000","last_clean":"2024-09-16T12:59:13.783361+0000","last_became_active":"2024-09-16T12:59:11.769106+0000","last_became_peered":"2024-09-16T12:59:11.769106+0000","last_unstale":"2024-09-16T12:59:13.783361+0000","last_undegraded":"2024-09-16T12:59:13.783361+0000","last_fullsized":"2024-09-16T12:59:13.783361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:20.022662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045226000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.777888+0000","last_change":"2024-09-16T12:59:01.688552+0000","last_active":"2024-09-16T12:59:13.777888+0000","last_peered":"2024-09-16T12:59:13.777888+0000","last_clean":"2024-09-16T12:59:13.777888+0000","last_became_active":"2024-09-16T12:59:01.688068+0000","last_became_peered":"2024-09-16T12:59:01.688068+0000","last_unstale":"2024-09-16T12:59:13.777888+0000","last_undegraded":"2024-09-16T12:59:13.777888+0000","last_fullsized":"2024-09-16T12:59:13.777888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:01.298345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.757289+0000","last_change":"2024-09-16T12:58:55.654478+0000","last_active":"2024-09-16T12:59:12.757289+0000","last_peered":"2024-09-16T12:59:12.757289+0000","last_clean":"2024-09-16T12:59:12.757289+0000","last_became_active":"2024-09-16T12:58:53.631689+0000","last_became_peered":"2024-09-16T12:58:53.631689+0000","last_unstale":"2024-09-16T12:59:12.757289+0000","last_undegraded":"2024-09-16T12:59:12.757289+0000","last_fullsized":"2024-09-16T12:59:12.757289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:28.863018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.784123+0000","last_change":"2024-09-16T12:59:13.784486+0000","last_active":"2024-09-16T12:59:13.784123+0000","last_peered":"2024-09-16T12:59:13.784123+0000","last_clean":"2024-09-16T12:59:13.784123+0000","last_became_active":"2024-09-16T12:59:11.766238+0000","last_became_peered":"2024-09-16T12:59:11.766238+0000","last_unstale":"2024-09-16T12:59:13.784123+0000","last_undegraded":"2024-09-16T12:59:13.784123+0000","last_fullsized":"2024-09-16T12:59:13.784123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:53.576802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077242,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.780627+0000","last_change":"2024-09-16T12:59:01.685865+0000","last_active":"2024-09-16T12:59:12.780627+0000","last_peered":"2024-09-16T12:59:12.780627+0000","last_clean":"2024-09-16T12:59:12.780627+0000","last_became_active":"2024-09-16T12:59:01.685545+0000","last_became_peered":"2024-09-16T12:59:01.685545+0000","last_unstale":"2024-09-16T12:59:12.780627+0000","last_undegraded":"2024-09-16T12:59:12.780627+0000","last_fullsized":"2024-09-16T12:59:12.780627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:18.484667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.768658+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T12:59:13.768658+0000","last_peered":"2024-09-16T12:59:13.768658+0000","last_clean":"2024-09-16T12:59:13.768658+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T12:59:13.768658+0000","last_undegraded":"2024-09-16T12:59:13.768658+0000","last_fullsized":"2024-09-16T12:59:13.768658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.784547+0000","last_change":"2024-09-16T12:59:13.784741+0000","last_active":"2024-09-16T12:59:13.784547+0000","last_peered":"2024-09-16T12:59:13.784547+0000","last_clean":"2024-09-16T12:59:13.784547+0000","last_became_active":"2024-09-16T12:59:11.770780+0000","last_became_peered":"2024-09-16T12:59:11.770780+0000","last_unstale":"2024-09-16T12:59:13.784547+0000","last_undegraded":"2024-09-16T12:59:13.784547+0000","last_fullsized":"2024-09-16T12:59:13.784547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:08:03.967900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064957100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.714350+0000","last_change":"2024-09-16T12:59:01.686741+0000","last_active":"2024-09-16T12:59:13.714350+0000","last_peered":"2024-09-16T12:59:13.714350+0000","last_clean":"2024-09-16T12:59:13.714350+0000","last_became_active":"2024-09-16T12:59:01.686267+0000","last_became_peered":"2024-09-16T12:59:01.686267+0000","last_unstale":"2024-09-16T12:59:13.714350+0000","last_undegraded":"2024-09-16T12:59:13.714350+0000","last_fullsized":"2024-09-16T12:59:13.714350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:49.310418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.717789+0000","last_change":"2024-09-16T12:58:56.031861+0000","last_active":"2024-09-16T12:59:13.717789+0000","last_peered":"2024-09-16T12:59:13.717789+0000","last_clean":"2024-09-16T12:59:13.717789+0000","last_became_active":"2024-09-16T12:58:53.635856+0000","last_became_peered":"2024-09-16T12:58:53.635856+0000","last_unstale":"2024-09-16T12:59:13.717789+0000","last_undegraded":"2024-09-16T12:59:13.717789+0000","last_fullsized":"2024-09-16T12:59:13.717789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:40.827808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.030843+0000","last_change":"2024-09-16T12:59:14.031075+0000","last_active":"2024-09-16T12:59:14.030843+0000","last_peered":"2024-09-16T12:59:14.030843+0000","last_clean":"2024-09-16T12:59:14.030843+0000","last_became_active":"2024-09-16T12:59:11.764318+0000","last_became_peered":"2024-09-16T12:59:11.764318+0000","last_unstale":"2024-09-16T12:59:14.030843+0000","last_undegraded":"2024-09-16T12:59:14.030843+0000","last_fullsized":"2024-09-16T12:59:14.030843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:40:38.548210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094804399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.754662+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T12:59:12.754662+0000","last_peered":"2024-09-16T12:59:12.754662+0000","last_clean":"2024-09-16T12:59:12.754662+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T12:59:12.754662+0000","last_undegraded":"2024-09-16T12:59:12.754662+0000","last_fullsized":"2024-09-16T12:59:12.754662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:20.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.757496+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T12:59:12.757496+0000","last_peered":"2024-09-16T12:59:12.757496+0000","last_clean":"2024-09-16T12:59:12.757496+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T12:59:12.757496+0000","last_undegraded":"2024-09-16T12:59:12.757496+0000","last_fullsized":"2024-09-16T12:59:12.757496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":114,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.717888+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T12:59:13.717888+0000","last_peered":"2024-09-16T12:59:13.717888+0000","last_clean":"2024-09-16T12:59:13.717888+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T12:59:13.717888+0000","last_undegraded":"2024-09-16T12:59:13.717888+0000","last_fullsized":"2024-09-16T12:59:13.717888+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.788788+0000","last_change":"2024-09-16T12:59:13.789075+0000","last_active":"2024-09-16T12:59:13.788788+0000","last_peered":"2024-09-16T12:59:13.788788+0000","last_clean":"2024-09-16T12:59:13.788788+0000","last_became_active":"2024-09-16T12:59:11.766779+0000","last_became_peered":"2024-09-16T12:59:11.766779+0000","last_unstale":"2024-09-16T12:59:13.788788+0000","last_undegraded":"2024-09-16T12:59:13.788788+0000","last_fullsized":"2024-09-16T12:59:13.788788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:47.829432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046786199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.795968+0000","last_change":"2024-09-16T12:59:13.796421+0000","last_active":"2024-09-16T12:59:13.795968+0000","last_peered":"2024-09-16T12:59:13.795968+0000","last_clean":"2024-09-16T12:59:13.795968+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T12:59:13.795968+0000","last_undegraded":"2024-09-16T12:59:13.795968+0000","last_fullsized":"2024-09-16T12:59:13.795968+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067313300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.778149+0000","last_change":"2024-09-16T12:59:13.778149+0000","last_active":"2024-09-16T12:59:13.778149+0000","last_peered":"2024-09-16T12:59:13.778149+0000","last_clean":"2024-09-16T12:59:13.778149+0000","last_became_active":"2024-09-16T12:59:11.770189+0000","last_became_peered":"2024-09-16T12:59:11.770189+0000","last_unstale":"2024-09-16T12:59:13.778149+0000","last_undegraded":"2024-09-16T12:59:13.778149+0000","last_fullsized":"2024-09-16T12:59:13.778149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:07:43.844099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011000280000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.789140+0000","last_change":"2024-09-16T12:59:13.789412+0000","last_active":"2024-09-16T12:59:13.789140+0000","last_peered":"2024-09-16T12:59:13.789140+0000","last_clean":"2024-09-16T12:59:13.789140+0000","last_became_active":"2024-09-16T12:59:11.767637+0000","last_became_peered":"2024-09-16T12:59:11.767637+0000","last_unstale":"2024-09-16T12:59:13.789140+0000","last_undegraded":"2024-09-16T12:59:13.789140+0000","last_fullsized":"2024-09-16T12:59:13.789140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:00.586099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078078700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.030230+0000","last_change":"2024-09-16T12:59:14.030458+0000","last_active":"2024-09-16T12:59:14.030230+0000","last_peered":"2024-09-16T12:59:14.030230+0000","last_clean":"2024-09-16T12:59:14.030230+0000","last_became_active":"2024-09-16T12:59:11.766582+0000","last_became_peered":"2024-09-16T12:59:11.766582+0000","last_unstale":"2024-09-16T12:59:14.030230+0000","last_undegraded":"2024-09-16T12:59:14.030230+0000","last_fullsized":"2024-09-16T12:59:14.030230+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:27.456165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053245600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.c","version":"34'2","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.743011+0000","last_change":"2024-09-16T12:59:13.780281+0000","last_active":"2024-09-16T12:59:14.743011+0000","last_peered":"2024-09-16T12:59:14.743011+0000","last_clean":"2024-09-16T12:59:14.743011+0000","last_became_active":"2024-09-16T12:59:11.764080+0000","last_became_peered":"2024-09-16T12:59:11.764080+0000","last_unstale":"2024-09-16T12:59:14.743011+0000","last_undegraded":"2024-09-16T12:59:14.743011+0000","last_fullsized":"2024-09-16T12:59:14.743011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:55:18.557326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020639640000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.780488+0000","last_change":"2024-09-16T12:59:13.780488+0000","last_active":"2024-09-16T12:59:13.780488+0000","last_peered":"2024-09-16T12:59:13.780488+0000","last_clean":"2024-09-16T12:59:13.780488+0000","last_became_active":"2024-09-16T12:59:11.764699+0000","last_became_peered":"2024-09-16T12:59:11.764699+0000","last_unstale":"2024-09-16T12:59:13.780488+0000","last_undegraded":"2024-09-16T12:59:13.780488+0000","last_fullsized":"2024-09-16T12:59:13.780488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:12.151943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001758094,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.780729+0000","last_change":"2024-09-16T12:59:13.780729+0000","last_active":"2024-09-16T12:59:13.780729+0000","last_peered":"2024-09-16T12:59:13.780729+0000","last_clean":"2024-09-16T12:59:13.780729+0000","last_became_active":"2024-09-16T12:59:11.770833+0000","last_became_peered":"2024-09-16T12:59:11.770833+0000","last_unstale":"2024-09-16T12:59:13.780729+0000","last_undegraded":"2024-09-16T12:59:13.780729+0000","last_fullsized":"2024-09-16T12:59:13.780729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:56.784122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637831,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.784178+0000","last_change":"2024-09-16T12:59:13.784503+0000","last_active":"2024-09-16T12:59:13.784178+0000","last_peered":"2024-09-16T12:59:13.784178+0000","last_clean":"2024-09-16T12:59:13.784178+0000","last_became_active":"2024-09-16T12:59:11.766018+0000","last_became_peered":"2024-09-16T12:59:11.766018+0000","last_unstale":"2024-09-16T12:59:13.784178+0000","last_undegraded":"2024-09-16T12:59:13.784178+0000","last_fullsized":"2024-09-16T12:59:13.784178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:06:11.268847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053943399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297728,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074112,"statfs":{"total":96636764160,"available":96331890688,"internally_reserved":0,"allocated":376832,"data_stored":151227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":376832,"data_stored":151216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301888,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069952,"statfs":{"total":96636764160,"available":96327630848,"internally_reserved":0,"allocated":376832,"data_stored":151230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":372736,"data_stored":151200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":380928,"data_stored":157367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]}]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301936,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069904,"statfs":{"total":96636764160,"available":96327581696,"internally_reserved":0,"allocated":425984,"data_stored":741549,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301920,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069920,"statfs":{"total":96636764160,"available":96327598080,"internally_reserved":0,"allocated":413696,"data_stored":734969,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":372736,"data_stored":151200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":49,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T12:59:18.263 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T12:59:18.274 INFO:teuthology.orchestra.run.smithi042.stdout:325 writesame 0xe2c7e4a thru 0xe2ca3fe data_size 0x7(0x25b5 bytes) 2024-09-16T12:59:18.274 INFO:teuthology.orchestra.run.smithi042.stdout:326 write 0xc2e04b6 thru 0xc2e2252 (0x1d9d bytes) 2024-09-16T12:59:18.302 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:18.302 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:18.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:18.302 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:18.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.302 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.667583188000002, 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006297965, 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629315+0000", 2024-09-16T12:59:18.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7080000000000002e-06 2024-09-16T12:59:18.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629324+0000", 2024-09-16T12:59:18.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0250000000000008e-06 2024-09-16T12:59:18.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629340+0000", 2024-09-16T12:59:18.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.59e-05 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629351+0000", 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1236999999999999e-05 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629565+0000", 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000213482 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629582+0000", 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7482999999999998e-05 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.631980+0000", 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023977830000000001 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.632247+0000", 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000266867 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.632356+0000", 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010894 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.633744+0000", 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013876240000000001 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635507+0000", 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001763288 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635562+0000", 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5096999999999998e-05 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635608+0000", 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5531000000000002e-05 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.309 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.655517590999999, 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063927960000000001, 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:18.310 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641378+0000", 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6429999999999999e-06 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641390+0000", 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1912e-05 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641422+0000", 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2666999999999997e-05 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641448+0000", 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5636e-05 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641623+0000", 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017479299999999999 2024-09-16T12:59:18.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641641+0000", 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8369999999999999e-05 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.644629+0000", 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00298745 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.644995+0000", 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000366688 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.645348+0000", 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035283200000000003 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647159+0000", 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018104760000000001 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647631+0000", 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047253799999999999 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647708+0000", 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6715e-05 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647768+0000", 2024-09-16T12:59:18.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0075999999999999e-05 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.647937632000001, 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006003988, 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.317 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 7 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:18.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648958+0000", 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0280000000000001e-06 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648970+0000", 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1374e-05 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649002+0000", 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2298999999999997e-05 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649014+0000", 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2442e-05 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649099+0000", 2024-09-16T12:59:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4655999999999997e-05 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649107+0000", 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0800000000000006e-06 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.652487+0000", 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003380241 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.652733+0000", 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024601000000000001 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.653091+0000", 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035755599999999998 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654603+0000", 2024-09-16T12:59:18.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001512163 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654795+0000", 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000192179 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654875+0000", 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0211999999999997e-05 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654959+0000", 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3747999999999998e-05 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:11 4.1f e34/31 4:fa654f18:::rbd_id.image_client.0:head v 34'1, mlcod=0'0)", 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:13.789305+0000", 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5075880470000005, 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051335770000000003, 2024-09-16T12:59:18.324 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789305+0000", 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789305+0000", 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789306+0000", 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1850000000000001e-06 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789317+0000", 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1344e-05 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789348+0000", 2024-09-16T12:59:18.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0545e-05 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789363+0000", 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.485e-05 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789438+0000", 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4884999999999999e-05 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789482+0000", 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4731999999999998e-05 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794352+0000", 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048695079999999998 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794438+0000", 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6527999999999996e-05 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.328 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:439 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.005732+0000", 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2911604620000001, 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050891779999999998, 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.329 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 439 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005732+0000", 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005732+0000", 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005734+0000", 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4139999999999999e-06 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005741+0000", 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.503e-06 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005768+0000", 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6982e-05 2024-09-16T12:59:18.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005787+0000", 2024-09-16T12:59:18.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8974000000000002e-05 2024-09-16T12:59:18.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.007568+0000", 2024-09-16T12:59:18.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017811070000000001 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.007851+0000", 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028264799999999998 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.008426+0000", 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057548500000000004 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.009913+0000", 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001486376 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.010615+0000", 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070175700000000001 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.010750+0000", 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135685 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.010822+0000", 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1247000000000004e-05 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:443 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.015364+0000", 2024-09-16T12:59:18.335 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.281529272, 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049269250000000004, 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 443 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.015364+0000", 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.015364+0000", 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.015364+0000", 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1699999999999997e-07 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.015369+0000", 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1999999999999996e-06 2024-09-16T12:59:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.015376+0000", 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8529999999999992e-06 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.015381+0000", 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9019999999999998e-06 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.017012+0000", 2024-09-16T12:59:18.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001630231 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.017263+0000", 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025172900000000002 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.018063+0000", 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00079936500000000001 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.019457+0000", 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001394436 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.020130+0000", 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067243299999999995 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.020221+0000", 2024-09-16T12:59:18.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1358000000000002e-05 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.020290+0000", 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9601e-05 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:489 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.117759+0000", 2024-09-16T12:59:18.341 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1791340260000001, 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046441490000000002, 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 489 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.117759+0000", 2024-09-16T12:59:18.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.117759+0000", 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.117759+0000", 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7100000000000002e-07 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.117762+0000", 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.362e-06 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.117777+0000", 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5551999999999999e-05 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.117781+0000", 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.738e-06 2024-09-16T12:59:18.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.118953+0000", 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011720249999999999 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.119203+0000", 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024957700000000001 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.119755+0000", 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055267000000000001 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.121025+0000", 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012699740000000001 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.122259+0000", 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012334760000000001 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.122332+0000", 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2867000000000006e-05 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.122403+0000", 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1336999999999998e-05 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:513 4.8 4:132f6b09:::rbd_data.11b59d02cc38.000000000000008c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.347 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1158002709999999, 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054902010000000001, 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 513 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:18.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181095+0000", 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4059999999999999e-06 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181106+0000", 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0868e-05 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181145+0000", 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9168000000000002e-05 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181157+0000", 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2343e-05 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181229+0000", 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1264000000000001e-05 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181415+0000", 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000186804 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181722+0000", 2024-09-16T12:59:18.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030624200000000003 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.185844+0000", 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041219359999999997 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186379+0000", 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053546599999999996 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186495+0000", 2024-09-16T12:59:18.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115575 2024-09-16T12:59:18.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186583+0000", 2024-09-16T12:59:18.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8128999999999996e-05 2024-09-16T12:59:18.356 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.356 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.356 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:532 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.219756+0000", 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.0771371059999999, 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051249149999999999, 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 532 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219756+0000", 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219756+0000", 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219758+0000", 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3099999999999999e-06 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219770+0000", 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1742999999999999e-05 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219805+0000", 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5102999999999999e-05 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219818+0000", 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219971+0000", 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015284000000000001 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.220132+0000", 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000161905 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.220456+0000", 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032376300000000002 2024-09-16T12:59:18.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.221604+0000", 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001148135 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.224727+0000", 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003122545 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.224827+0000", 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9901000000000005e-05 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.224881+0000", 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3786999999999998e-05 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:542 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:18.362 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.0628606490000001, 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060375849999999998, 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 542 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234033+0000", 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8599999999999996e-07 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234038+0000", 2024-09-16T12:59:18.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4830000000000001e-06 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234047+0000", 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3109999999999995e-06 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234053+0000", 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6339999999999998e-06 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.235355+0000", 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013027589999999999 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.235617+0000", 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026182000000000001 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.236547+0000", 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093014599999999997 2024-09-16T12:59:18.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239456+0000", 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029083289999999999 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239788+0000", 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033232900000000002 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239946+0000", 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015829700000000001 2024-09-16T12:59:18.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.240070+0000", 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012349099999999999 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:555 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.262998+0000", 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.03389485, 2024-09-16T12:59:18.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046909300000000003, 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 555 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.262998+0000", 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.262998+0000", 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.262999+0000", 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.229e-06 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.263006+0000", 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7769999999999997e-06 2024-09-16T12:59:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.263038+0000", 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2422000000000002e-05 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.263049+0000", 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0550999999999999e-05 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.264343+0000", 2024-09-16T12:59:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012940639999999999 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.264483+0000", 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014026500000000001 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.265233+0000", 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074938700000000001 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.267172+0000", 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019392439999999999 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.267512+0000", 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000340129 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.267583+0000", 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1441999999999998e-05 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.267689+0000", 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010542 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:571 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.987828759, 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005551762, 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 571 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:18.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309066+0000", 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2060000000000001e-06 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309080+0000", 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3413e-05 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309114+0000", 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4180000000000001e-05 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309126+0000", 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2129e-05 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309312+0000", 2024-09-16T12:59:18.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018575800000000001 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309477+0000", 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000164885 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.310054+0000", 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057696199999999998 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.311809+0000", 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001755047 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314482+0000", 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026735330000000001 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314559+0000", 2024-09-16T12:59:18.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6884000000000005e-05 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314616+0000", 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6765000000000003e-05 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:589 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.335679+0000", 2024-09-16T12:59:18.380 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.96121404099999996, 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050963889999999998, 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 589 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335679+0000", 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335679+0000", 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335680+0000", 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5700000000000001e-07 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335684+0000", 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6149999999999999e-06 2024-09-16T12:59:18.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335697+0000", 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2465e-05 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335703+0000", 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7870000000000004e-06 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.336921+0000", 2024-09-16T12:59:18.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001217625 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.337168+0000", 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024640300000000002 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.337796+0000", 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00062875099999999998 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340375+0000", 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025782579999999999 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340644+0000", 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026920399999999999 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340712+0000", 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639000000000005e-05 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340775+0000", 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2785000000000003e-05 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:634 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.470034+0000", 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.82685907000000003, 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045022969999999997, 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 634 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.470034+0000", 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.470034+0000", 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.470035+0000", 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4200000000000005e-07 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.470045+0000", 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0059e-05 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.470058+0000", 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3558e-05 2024-09-16T12:59:18.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.470070+0000", 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1629e-05 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.471276+0000", 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001206525 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.471480+0000", 2024-09-16T12:59:18.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020343 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.472225+0000", 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074553800000000002 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473619+0000", 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013938310000000001 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.474338+0000", 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000719026 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.474451+0000", 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112629 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.474536+0000", 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5229999999999995e-05 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.391 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:638 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.79365719599999995, 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052595139999999999, 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.392 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 638 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503237+0000", 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.02e-06 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503248+0000", 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1114000000000001e-05 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503263+0000", 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092999999999999e-05 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503273+0000", 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0413e-05 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.504835+0000", 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001562164 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.505031+0000", 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000195465 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.505731+0000", 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070030899999999998 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507931+0000", 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022001429999999999 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508299+0000", 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036765399999999997 2024-09-16T12:59:18.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508405+0000", 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106368 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508495+0000", 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9771000000000005e-05 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.397 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:746 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.52775745100000004, 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052549399999999996, 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 746 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:18.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769137+0000", 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330000000000001e-06 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769144+0000", 2024-09-16T12:59:18.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2660000000000004e-06 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769162+0000", 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7997000000000001e-05 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769178+0000", 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6305999999999999e-05 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.770704+0000", 2024-09-16T12:59:18.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015260880000000001 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.770965+0000", 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026027800000000002 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.771962+0000", 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099762399999999991 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.773232+0000", 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012702499999999999 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774136+0000", 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000903355 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774275+0000", 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139431 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774390+0000", 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011511200000000001 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:771 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.819598+0000", 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.47729522000000002, 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046261879999999998, 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 771 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.819598+0000", 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.819598+0000", 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.819599+0000", 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8759999999999999e-06 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.819608+0000", 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3100000000000001e-06 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.819630+0000", 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2645000000000001e-05 2024-09-16T12:59:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.819642+0000", 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1598e-05 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.819712+0000", 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0151e-05 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.819718+0000", 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5500000000000002e-06 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.822222+0000", 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025040549999999998 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.822376+0000", 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000154174 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.822821+0000", 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000444585 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.823935+0000", 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011143940000000001 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.824125+0000", 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000189896 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.824181+0000", 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6456999999999998e-05 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.824224+0000", 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2497000000000003e-05 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:820 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.917951+0000", 2024-09-16T12:59:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.37894165600000002, 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046321130000000002, 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 820 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.917951+0000", 2024-09-16T12:59:18.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.917951+0000", 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.917953+0000", 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2029999999999999e-06 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.917965+0000", 2024-09-16T12:59:18.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2006e-05 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.917999+0000", 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3623e-05 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918016+0000", 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.736e-05 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918127+0000", 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110314 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918281+0000", 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015399399999999999 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918813+0000", 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053222600000000001 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.922120+0000", 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033073809999999999 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.922226+0000", 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105654 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.922434+0000", 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020837500000000001 2024-09-16T12:59:18.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.922583+0000", 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014897700000000001 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:821 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.37884521500000001, 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052264670000000003, 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.415 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 821 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918049+0000", 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.734e-06 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918058+0000", 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918078+0000", 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0163e-05 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918100+0000", 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1396999999999999e-05 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.919700+0000", 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001600838 2024-09-16T12:59:18.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.919878+0000", 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177276 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.920899+0000", 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010218219999999999 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.922776+0000", 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001876436 2024-09-16T12:59:18.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923007+0000", 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023069799999999999 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923160+0000", 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015317299999999999 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923274+0000", 2024-09-16T12:59:18.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114298 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:843 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.961705+0000", 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.33518757199999999, 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049882310000000001, 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 843 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961705+0000", 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961705+0000", 2024-09-16T12:59:18.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961706+0000", 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6200000000000006e-07 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961710+0000", 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8619999999999996e-06 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961718+0000", 2024-09-16T12:59:18.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7999999999999999e-06 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961723+0000", 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6859999999999997e-06 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.963536+0000", 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018133240000000001 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.963693+0000", 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015719 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.964589+0000", 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089570199999999996 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.965980+0000", 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001391572 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.966585+0000", 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060418400000000001 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.966646+0000", 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1038999999999996e-05 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.966693+0000", 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.791e-05 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.426 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.427 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.427 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.427 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.427 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:18.429 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:18.429 INFO:teuthology.orchestra.run.smithi042.stdout:327 trunc from 0xe2ca3ff to 0x6dd665 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:328 write 0x174707d thru 0x174e374 (0x72f8 bytes) 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:329 write 0x106bb82 thru 0x106f29d (0x371c bytes) 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:331 write 0x30eee99 thru 0x30fe2bc (0xf424 bytes) 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:332 trunc from 0x30fe2bd to 0x491138c 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:336 read 0x1f882df thru 0x1f916c8 (0x93ea bytes) 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:337 read 0x2dc9d44 thru 0x2dce2a3 (0x4560 bytes) 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:342 punch from 0x3560280 to 0x356f804, (0xf584 bytes) 2024-09-16T12:59:18.430 INFO:teuthology.orchestra.run.smithi042.stdout:343 writesame 0x97ef153 thru 0x97effe0 data_size 0x6(0xe8e bytes) 2024-09-16T12:59:18.430 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:18.423+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout:344 writesame 0xdcf9217 thru 0xdcf9811 data_size 0x1(0x5fb bytes) 2024-09-16T12:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout:346 write 0x7979970 thru 0x79878d4 (0xdf65 bytes) 2024-09-16T12:59:18.443 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:18.443 INFO:teuthology.orchestra.run.smithi042.stdout:350 writesame 0xac44096 thru 0xac4ccf7 data_size 0xe(0x8c62 bytes) 2024-09-16T12:59:18.461 INFO:teuthology.orchestra.run.smithi042.stdout:351 write 0xe3bd209 thru 0xe3c83e6 (0xb1de bytes) 2024-09-16T12:59:18.468 INFO:teuthology.orchestra.run.smithi042.stdout:352 write 0x55f9435 thru 0x55fa2b0 (0xe7c bytes) 2024-09-16T12:59:18.473 INFO:teuthology.orchestra.run.smithi042.stdout:353 read 0x4468af4 thru 0x446cca7 (0x41b4 bytes) 2024-09-16T12:59:18.473 INFO:teuthology.orchestra.run.smithi042.stdout:356 read 0xdbb06c thru 0xdc1169 (0x60fe bytes) 2024-09-16T12:59:18.473 INFO:teuthology.orchestra.run.smithi042.stdout:357 clone 2 order 20 su 8192 sc 7 2024-09-16T12:59:18.529 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:18.523+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:18.537 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:18.537 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:18.537 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:18.537 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:18.546 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:59:18.580 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T12:59:18.593 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-16T12:59:18.240404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":100,"num_read_kb":81,"num_write":109,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411016,"kb_used_data":3024,"kb_used_omap":13,"kb_used_meta":2407858,"kb_avail":752563704,"statfs":{"total":773094113280,"available":770625232896,"internally_reserved":0,"allocated":3096576,"data_stored":2389958,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2465647296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.008488"},"pg_stats":[{"pgid":"4.1f","version":"34'1","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.804599+0000","last_change":"2024-09-16T12:59:13.789786+0000","last_active":"2024-09-16T12:59:13.804599+0000","last_peered":"2024-09-16T12:59:13.804599+0000","last_clean":"2024-09-16T12:59:13.804599+0000","last_became_active":"2024-09-16T12:59:11.766798+0000","last_became_peered":"2024-09-16T12:59:11.766798+0000","last_unstale":"2024-09-16T12:59:13.804599+0000","last_undegraded":"2024-09-16T12:59:13.804599+0000","last_fullsized":"2024-09-16T12:59:13.804599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:22.588888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.030274+0000","last_change":"2024-09-16T12:59:14.030480+0000","last_active":"2024-09-16T12:59:14.030274+0000","last_peered":"2024-09-16T12:59:14.030274+0000","last_clean":"2024-09-16T12:59:14.030274+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T12:59:14.030274+0000","last_undegraded":"2024-09-16T12:59:14.030274+0000","last_fullsized":"2024-09-16T12:59:14.030274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049602200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.769948+0000","last_change":"2024-09-16T12:59:13.770162+0000","last_active":"2024-09-16T12:59:13.769948+0000","last_peered":"2024-09-16T12:59:13.769948+0000","last_clean":"2024-09-16T12:59:13.769948+0000","last_became_active":"2024-09-16T12:59:11.761823+0000","last_became_peered":"2024-09-16T12:59:11.761823+0000","last_unstale":"2024-09-16T12:59:13.769948+0000","last_undegraded":"2024-09-16T12:59:13.769948+0000","last_fullsized":"2024-09-16T12:59:13.769948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:33.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012332339999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1c","version":"34'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.782901+0000","last_change":"2024-09-16T12:59:13.770236+0000","last_active":"2024-09-16T12:59:13.782901+0000","last_peered":"2024-09-16T12:59:13.782901+0000","last_clean":"2024-09-16T12:59:13.782901+0000","last_became_active":"2024-09-16T12:59:11.766604+0000","last_became_peered":"2024-09-16T12:59:11.766604+0000","last_unstale":"2024-09-16T12:59:13.782901+0000","last_undegraded":"2024-09-16T12:59:13.782901+0000","last_fullsized":"2024-09-16T12:59:13.782901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:01.891108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015138090000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.779899+0000","last_change":"2024-09-16T12:59:13.779899+0000","last_active":"2024-09-16T12:59:13.779899+0000","last_peered":"2024-09-16T12:59:13.779899+0000","last_clean":"2024-09-16T12:59:13.779899+0000","last_became_active":"2024-09-16T12:59:11.759998+0000","last_became_peered":"2024-09-16T12:59:11.759998+0000","last_unstale":"2024-09-16T12:59:13.779899+0000","last_undegraded":"2024-09-16T12:59:13.779899+0000","last_fullsized":"2024-09-16T12:59:13.779899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:49:22.613940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002573915,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1a","version":"34'2","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.814406+0000","last_change":"2024-09-16T12:59:13.779650+0000","last_active":"2024-09-16T12:59:13.814406+0000","last_peered":"2024-09-16T12:59:13.814406+0000","last_clean":"2024-09-16T12:59:13.814406+0000","last_became_active":"2024-09-16T12:59:11.759751+0000","last_became_peered":"2024-09-16T12:59:11.759751+0000","last_unstale":"2024-09-16T12:59:13.814406+0000","last_undegraded":"2024-09-16T12:59:13.814406+0000","last_fullsized":"2024-09-16T12:59:13.814406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:12.049622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002565964,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.770227+0000","last_change":"2024-09-16T12:59:13.770391+0000","last_active":"2024-09-16T12:59:13.770227+0000","last_peered":"2024-09-16T12:59:13.770227+0000","last_clean":"2024-09-16T12:59:13.770227+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T12:59:13.770227+0000","last_undegraded":"2024-09-16T12:59:13.770227+0000","last_fullsized":"2024-09-16T12:59:13.770227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013782849999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.777900+0000","last_change":"2024-09-16T12:59:13.777900+0000","last_active":"2024-09-16T12:59:13.777900+0000","last_peered":"2024-09-16T12:59:13.777900+0000","last_clean":"2024-09-16T12:59:13.777900+0000","last_became_active":"2024-09-16T12:59:11.767308+0000","last_became_peered":"2024-09-16T12:59:11.767308+0000","last_unstale":"2024-09-16T12:59:13.777900+0000","last_undegraded":"2024-09-16T12:59:13.777900+0000","last_fullsized":"2024-09-16T12:59:13.777900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:40.593369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093274999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.770445+0000","last_change":"2024-09-16T12:59:13.770695+0000","last_active":"2024-09-16T12:59:13.770445+0000","last_peered":"2024-09-16T12:59:13.770445+0000","last_clean":"2024-09-16T12:59:13.770445+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T12:59:13.770445+0000","last_undegraded":"2024-09-16T12:59:13.770445+0000","last_fullsized":"2024-09-16T12:59:13.770445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001481132,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.030536+0000","last_change":"2024-09-16T12:59:14.030762+0000","last_active":"2024-09-16T12:59:14.030536+0000","last_peered":"2024-09-16T12:59:14.030536+0000","last_clean":"2024-09-16T12:59:14.030536+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T12:59:14.030536+0000","last_undegraded":"2024-09-16T12:59:14.030536+0000","last_fullsized":"2024-09-16T12:59:14.030536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685533,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.796139+0000","last_change":"2024-09-16T12:59:13.796422+0000","last_active":"2024-09-16T12:59:13.796139+0000","last_peered":"2024-09-16T12:59:13.796139+0000","last_clean":"2024-09-16T12:59:13.796139+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T12:59:13.796139+0000","last_undegraded":"2024-09-16T12:59:13.796139+0000","last_fullsized":"2024-09-16T12:59:13.796139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055065799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.770518+0000","last_change":"2024-09-16T12:59:13.770810+0000","last_active":"2024-09-16T12:59:13.770518+0000","last_peered":"2024-09-16T12:59:13.770518+0000","last_clean":"2024-09-16T12:59:13.770518+0000","last_became_active":"2024-09-16T12:59:11.767626+0000","last_became_peered":"2024-09-16T12:59:11.767626+0000","last_unstale":"2024-09-16T12:59:13.770518+0000","last_undegraded":"2024-09-16T12:59:13.770518+0000","last_fullsized":"2024-09-16T12:59:13.770518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:13.669310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013482349999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.13","version":"34'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.800199+0000","last_change":"2024-09-16T12:59:13.780086+0000","last_active":"2024-09-16T12:59:13.800199+0000","last_peered":"2024-09-16T12:59:13.800199+0000","last_clean":"2024-09-16T12:59:13.800199+0000","last_became_active":"2024-09-16T12:59:11.770883+0000","last_became_peered":"2024-09-16T12:59:11.770883+0000","last_unstale":"2024-09-16T12:59:13.800199+0000","last_undegraded":"2024-09-16T12:59:13.800199+0000","last_fullsized":"2024-09-16T12:59:13.800199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:22.020502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024518000000000001,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.783323+0000","last_change":"2024-09-16T12:59:13.783659+0000","last_active":"2024-09-16T12:59:13.783323+0000","last_peered":"2024-09-16T12:59:13.783323+0000","last_clean":"2024-09-16T12:59:13.783323+0000","last_became_active":"2024-09-16T12:59:11.768690+0000","last_became_peered":"2024-09-16T12:59:11.768690+0000","last_unstale":"2024-09-16T12:59:13.783323+0000","last_undegraded":"2024-09-16T12:59:13.783323+0000","last_fullsized":"2024-09-16T12:59:13.783323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:00.837257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061900799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.769884+0000","last_change":"2024-09-16T12:59:13.770311+0000","last_active":"2024-09-16T12:59:13.769884+0000","last_peered":"2024-09-16T12:59:13.769884+0000","last_clean":"2024-09-16T12:59:13.769884+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T12:59:13.769884+0000","last_undegraded":"2024-09-16T12:59:13.769884+0000","last_fullsized":"2024-09-16T12:59:13.769884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413508,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.769309+0000","last_change":"2024-09-16T12:59:13.769620+0000","last_active":"2024-09-16T12:59:13.769309+0000","last_peered":"2024-09-16T12:59:13.769309+0000","last_clean":"2024-09-16T12:59:13.769309+0000","last_became_active":"2024-09-16T12:59:11.763566+0000","last_became_peered":"2024-09-16T12:59:11.763566+0000","last_unstale":"2024-09-16T12:59:13.769309+0000","last_undegraded":"2024-09-16T12:59:13.769309+0000","last_fullsized":"2024-09-16T12:59:13.769309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:16.800665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047952,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.119623+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T12:59:13.119623+0000","last_peered":"2024-09-16T12:59:13.119623+0000","last_clean":"2024-09-16T12:59:13.119623+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T12:59:13.119623+0000","last_undegraded":"2024-09-16T12:59:13.119623+0000","last_fullsized":"2024-09-16T12:59:13.119623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.777645+0000","last_change":"2024-09-16T12:59:13.777645+0000","last_active":"2024-09-16T12:59:13.777645+0000","last_peered":"2024-09-16T12:59:13.777645+0000","last_clean":"2024-09-16T12:59:13.777645+0000","last_became_active":"2024-09-16T12:59:11.770325+0000","last_became_peered":"2024-09-16T12:59:11.770325+0000","last_unstale":"2024-09-16T12:59:13.777645+0000","last_undegraded":"2024-09-16T12:59:13.777645+0000","last_fullsized":"2024-09-16T12:59:13.777645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:06:31.319709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069048899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.781099+0000","last_change":"2024-09-16T12:58:55.653750+0000","last_active":"2024-09-16T12:59:12.781099+0000","last_peered":"2024-09-16T12:59:12.781099+0000","last_clean":"2024-09-16T12:59:12.781099+0000","last_became_active":"2024-09-16T12:58:53.634053+0000","last_became_peered":"2024-09-16T12:58:53.634053+0000","last_unstale":"2024-09-16T12:59:12.781099+0000","last_undegraded":"2024-09-16T12:59:12.781099+0000","last_fullsized":"2024-09-16T12:59:12.781099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:50.402394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065616299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.769355+0000","last_change":"2024-09-16T12:59:13.769781+0000","last_active":"2024-09-16T12:59:13.769355+0000","last_peered":"2024-09-16T12:59:13.769355+0000","last_clean":"2024-09-16T12:59:13.769355+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T12:59:13.769355+0000","last_undegraded":"2024-09-16T12:59:13.769355+0000","last_fullsized":"2024-09-16T12:59:13.769355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010481990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.717846+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T12:59:13.717846+0000","last_peered":"2024-09-16T12:59:13.717846+0000","last_clean":"2024-09-16T12:59:13.717846+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T12:59:13.717846+0000","last_undegraded":"2024-09-16T12:59:13.717846+0000","last_fullsized":"2024-09-16T12:59:13.717846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.030580+0000","last_change":"2024-09-16T12:59:14.030767+0000","last_active":"2024-09-16T12:59:14.030580+0000","last_peered":"2024-09-16T12:59:14.030580+0000","last_clean":"2024-09-16T12:59:14.030580+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T12:59:14.030580+0000","last_undegraded":"2024-09-16T12:59:14.030580+0000","last_fullsized":"2024-09-16T12:59:14.030580+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000670212,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.780996+0000","last_change":"2024-09-16T12:58:55.653753+0000","last_active":"2024-09-16T12:59:12.780996+0000","last_peered":"2024-09-16T12:59:12.780996+0000","last_clean":"2024-09-16T12:59:12.780996+0000","last_became_active":"2024-09-16T12:58:53.630727+0000","last_became_peered":"2024-09-16T12:58:53.630727+0000","last_unstale":"2024-09-16T12:59:12.780996+0000","last_undegraded":"2024-09-16T12:59:12.780996+0000","last_fullsized":"2024-09-16T12:59:12.780996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:00.855732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061053599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.783361+0000","last_change":"2024-09-16T12:59:13.783668+0000","last_active":"2024-09-16T12:59:13.783361+0000","last_peered":"2024-09-16T12:59:13.783361+0000","last_clean":"2024-09-16T12:59:13.783361+0000","last_became_active":"2024-09-16T12:59:11.769106+0000","last_became_peered":"2024-09-16T12:59:11.769106+0000","last_unstale":"2024-09-16T12:59:13.783361+0000","last_undegraded":"2024-09-16T12:59:13.783361+0000","last_fullsized":"2024-09-16T12:59:13.783361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:20.022662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045226000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.777888+0000","last_change":"2024-09-16T12:59:01.688552+0000","last_active":"2024-09-16T12:59:13.777888+0000","last_peered":"2024-09-16T12:59:13.777888+0000","last_clean":"2024-09-16T12:59:13.777888+0000","last_became_active":"2024-09-16T12:59:01.688068+0000","last_became_peered":"2024-09-16T12:59:01.688068+0000","last_unstale":"2024-09-16T12:59:13.777888+0000","last_undegraded":"2024-09-16T12:59:13.777888+0000","last_fullsized":"2024-09-16T12:59:13.777888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:01.298345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.757289+0000","last_change":"2024-09-16T12:58:55.654478+0000","last_active":"2024-09-16T12:59:12.757289+0000","last_peered":"2024-09-16T12:59:12.757289+0000","last_clean":"2024-09-16T12:59:12.757289+0000","last_became_active":"2024-09-16T12:58:53.631689+0000","last_became_peered":"2024-09-16T12:58:53.631689+0000","last_unstale":"2024-09-16T12:59:12.757289+0000","last_undegraded":"2024-09-16T12:59:12.757289+0000","last_fullsized":"2024-09-16T12:59:12.757289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:28.863018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.784123+0000","last_change":"2024-09-16T12:59:13.784486+0000","last_active":"2024-09-16T12:59:13.784123+0000","last_peered":"2024-09-16T12:59:13.784123+0000","last_clean":"2024-09-16T12:59:13.784123+0000","last_became_active":"2024-09-16T12:59:11.766238+0000","last_became_peered":"2024-09-16T12:59:11.766238+0000","last_unstale":"2024-09-16T12:59:13.784123+0000","last_undegraded":"2024-09-16T12:59:13.784123+0000","last_fullsized":"2024-09-16T12:59:13.784123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:53.576802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077242,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.780627+0000","last_change":"2024-09-16T12:59:01.685865+0000","last_active":"2024-09-16T12:59:12.780627+0000","last_peered":"2024-09-16T12:59:12.780627+0000","last_clean":"2024-09-16T12:59:12.780627+0000","last_became_active":"2024-09-16T12:59:01.685545+0000","last_became_peered":"2024-09-16T12:59:01.685545+0000","last_unstale":"2024-09-16T12:59:12.780627+0000","last_undegraded":"2024-09-16T12:59:12.780627+0000","last_fullsized":"2024-09-16T12:59:12.780627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:18.484667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.768658+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T12:59:13.768658+0000","last_peered":"2024-09-16T12:59:13.768658+0000","last_clean":"2024-09-16T12:59:13.768658+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T12:59:13.768658+0000","last_undegraded":"2024-09-16T12:59:13.768658+0000","last_fullsized":"2024-09-16T12:59:13.768658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.784547+0000","last_change":"2024-09-16T12:59:13.784741+0000","last_active":"2024-09-16T12:59:13.784547+0000","last_peered":"2024-09-16T12:59:13.784547+0000","last_clean":"2024-09-16T12:59:13.784547+0000","last_became_active":"2024-09-16T12:59:11.770780+0000","last_became_peered":"2024-09-16T12:59:11.770780+0000","last_unstale":"2024-09-16T12:59:13.784547+0000","last_undegraded":"2024-09-16T12:59:13.784547+0000","last_fullsized":"2024-09-16T12:59:13.784547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:08:03.967900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064957100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.714350+0000","last_change":"2024-09-16T12:59:01.686741+0000","last_active":"2024-09-16T12:59:13.714350+0000","last_peered":"2024-09-16T12:59:13.714350+0000","last_clean":"2024-09-16T12:59:13.714350+0000","last_became_active":"2024-09-16T12:59:01.686267+0000","last_became_peered":"2024-09-16T12:59:01.686267+0000","last_unstale":"2024-09-16T12:59:13.714350+0000","last_undegraded":"2024-09-16T12:59:13.714350+0000","last_fullsized":"2024-09-16T12:59:13.714350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:49.310418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.717789+0000","last_change":"2024-09-16T12:58:56.031861+0000","last_active":"2024-09-16T12:59:13.717789+0000","last_peered":"2024-09-16T12:59:13.717789+0000","last_clean":"2024-09-16T12:59:13.717789+0000","last_became_active":"2024-09-16T12:58:53.635856+0000","last_became_peered":"2024-09-16T12:58:53.635856+0000","last_unstale":"2024-09-16T12:59:13.717789+0000","last_undegraded":"2024-09-16T12:59:13.717789+0000","last_fullsized":"2024-09-16T12:59:13.717789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:40.827808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.030843+0000","last_change":"2024-09-16T12:59:14.031075+0000","last_active":"2024-09-16T12:59:14.030843+0000","last_peered":"2024-09-16T12:59:14.030843+0000","last_clean":"2024-09-16T12:59:14.030843+0000","last_became_active":"2024-09-16T12:59:11.764318+0000","last_became_peered":"2024-09-16T12:59:11.764318+0000","last_unstale":"2024-09-16T12:59:14.030843+0000","last_undegraded":"2024-09-16T12:59:14.030843+0000","last_fullsized":"2024-09-16T12:59:14.030843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:40:38.548210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094804399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.754662+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T12:59:12.754662+0000","last_peered":"2024-09-16T12:59:12.754662+0000","last_clean":"2024-09-16T12:59:12.754662+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T12:59:12.754662+0000","last_undegraded":"2024-09-16T12:59:12.754662+0000","last_fullsized":"2024-09-16T12:59:12.754662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:20.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:12.757496+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T12:59:12.757496+0000","last_peered":"2024-09-16T12:59:12.757496+0000","last_clean":"2024-09-16T12:59:12.757496+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T12:59:12.757496+0000","last_undegraded":"2024-09-16T12:59:12.757496+0000","last_fullsized":"2024-09-16T12:59:12.757496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":114,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.717888+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T12:59:13.717888+0000","last_peered":"2024-09-16T12:59:13.717888+0000","last_clean":"2024-09-16T12:59:13.717888+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T12:59:13.717888+0000","last_undegraded":"2024-09-16T12:59:13.717888+0000","last_fullsized":"2024-09-16T12:59:13.717888+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.788788+0000","last_change":"2024-09-16T12:59:13.789075+0000","last_active":"2024-09-16T12:59:13.788788+0000","last_peered":"2024-09-16T12:59:13.788788+0000","last_clean":"2024-09-16T12:59:13.788788+0000","last_became_active":"2024-09-16T12:59:11.766779+0000","last_became_peered":"2024-09-16T12:59:11.766779+0000","last_unstale":"2024-09-16T12:59:13.788788+0000","last_undegraded":"2024-09-16T12:59:13.788788+0000","last_fullsized":"2024-09-16T12:59:13.788788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:47.829432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046786199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.795968+0000","last_change":"2024-09-16T12:59:13.796421+0000","last_active":"2024-09-16T12:59:13.795968+0000","last_peered":"2024-09-16T12:59:13.795968+0000","last_clean":"2024-09-16T12:59:13.795968+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T12:59:13.795968+0000","last_undegraded":"2024-09-16T12:59:13.795968+0000","last_fullsized":"2024-09-16T12:59:13.795968+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067313300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.778149+0000","last_change":"2024-09-16T12:59:13.778149+0000","last_active":"2024-09-16T12:59:13.778149+0000","last_peered":"2024-09-16T12:59:13.778149+0000","last_clean":"2024-09-16T12:59:13.778149+0000","last_became_active":"2024-09-16T12:59:11.770189+0000","last_became_peered":"2024-09-16T12:59:11.770189+0000","last_unstale":"2024-09-16T12:59:13.778149+0000","last_undegraded":"2024-09-16T12:59:13.778149+0000","last_fullsized":"2024-09-16T12:59:13.778149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:07:43.844099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011000280000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.789140+0000","last_change":"2024-09-16T12:59:13.789412+0000","last_active":"2024-09-16T12:59:13.789140+0000","last_peered":"2024-09-16T12:59:13.789140+0000","last_clean":"2024-09-16T12:59:13.789140+0000","last_became_active":"2024-09-16T12:59:11.767637+0000","last_became_peered":"2024-09-16T12:59:11.767637+0000","last_unstale":"2024-09-16T12:59:13.789140+0000","last_undegraded":"2024-09-16T12:59:13.789140+0000","last_fullsized":"2024-09-16T12:59:13.789140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:00.586099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078078700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.030230+0000","last_change":"2024-09-16T12:59:14.030458+0000","last_active":"2024-09-16T12:59:14.030230+0000","last_peered":"2024-09-16T12:59:14.030230+0000","last_clean":"2024-09-16T12:59:14.030230+0000","last_became_active":"2024-09-16T12:59:11.766582+0000","last_became_peered":"2024-09-16T12:59:11.766582+0000","last_unstale":"2024-09-16T12:59:14.030230+0000","last_undegraded":"2024-09-16T12:59:14.030230+0000","last_fullsized":"2024-09-16T12:59:14.030230+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:27.456165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053245600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.c","version":"34'2","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:14.743011+0000","last_change":"2024-09-16T12:59:13.780281+0000","last_active":"2024-09-16T12:59:14.743011+0000","last_peered":"2024-09-16T12:59:14.743011+0000","last_clean":"2024-09-16T12:59:14.743011+0000","last_became_active":"2024-09-16T12:59:11.764080+0000","last_became_peered":"2024-09-16T12:59:11.764080+0000","last_unstale":"2024-09-16T12:59:14.743011+0000","last_undegraded":"2024-09-16T12:59:14.743011+0000","last_fullsized":"2024-09-16T12:59:14.743011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:55:18.557326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020639640000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.780488+0000","last_change":"2024-09-16T12:59:13.780488+0000","last_active":"2024-09-16T12:59:13.780488+0000","last_peered":"2024-09-16T12:59:13.780488+0000","last_clean":"2024-09-16T12:59:13.780488+0000","last_became_active":"2024-09-16T12:59:11.764699+0000","last_became_peered":"2024-09-16T12:59:11.764699+0000","last_unstale":"2024-09-16T12:59:13.780488+0000","last_undegraded":"2024-09-16T12:59:13.780488+0000","last_fullsized":"2024-09-16T12:59:13.780488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:12.151943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001758094,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.780729+0000","last_change":"2024-09-16T12:59:13.780729+0000","last_active":"2024-09-16T12:59:13.780729+0000","last_peered":"2024-09-16T12:59:13.780729+0000","last_clean":"2024-09-16T12:59:13.780729+0000","last_became_active":"2024-09-16T12:59:11.770833+0000","last_became_peered":"2024-09-16T12:59:11.770833+0000","last_unstale":"2024-09-16T12:59:13.780729+0000","last_undegraded":"2024-09-16T12:59:13.780729+0000","last_fullsized":"2024-09-16T12:59:13.780729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:56.784122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637831,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T12:59:13.784178+0000","last_change":"2024-09-16T12:59:13.784503+0000","last_active":"2024-09-16T12:59:13.784178+0000","last_peered":"2024-09-16T12:59:13.784178+0000","last_clean":"2024-09-16T12:59:13.784178+0000","last_became_active":"2024-09-16T12:59:11.766018+0000","last_became_peered":"2024-09-16T12:59:11.766018+0000","last_unstale":"2024-09-16T12:59:13.784178+0000","last_undegraded":"2024-09-16T12:59:13.784178+0000","last_fullsized":"2024-09-16T12:59:13.784178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:06:11.268847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053943399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297728,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074112,"statfs":{"total":96636764160,"available":96331890688,"internally_reserved":0,"allocated":376832,"data_stored":151227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":376832,"data_stored":151216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301888,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069952,"statfs":{"total":96636764160,"available":96327630848,"internally_reserved":0,"allocated":376832,"data_stored":151230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":372736,"data_stored":151200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":380928,"data_stored":157367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]}]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301936,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069904,"statfs":{"total":96636764160,"available":96327581696,"internally_reserved":0,"allocated":425984,"data_stored":741549,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301920,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069920,"statfs":{"total":96636764160,"available":96327598080,"internally_reserved":0,"allocated":413696,"data_stored":734969,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301884,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069956,"statfs":{"total":96636764160,"available":96327634944,"internally_reserved":0,"allocated":372736,"data_stored":151200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":49,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T12:59:18.595 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T12:59:18.595 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T12:59:18.595 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T12:59:18.596 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-16T12:59:18.630 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:18.627+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:18.651 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:18.652 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:18.660 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:18.731 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:18.727+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9806905849999996, 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089374989999999998, 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 11 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785978+0000", 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.373e-06 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785989+0000", 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1596999999999999e-05 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786008+0000", 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8510999999999999e-05 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786016+0000", 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786102+0000", 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6339000000000001e-05 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786113+0000", 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1386000000000001e-05 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.788774+0000", 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026609519999999998 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.788888+0000", 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113189 2024-09-16T12:59:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789043+0000", 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015570800000000001 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794705+0000", 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056619640000000002 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794770+0000", 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4789999999999995e-05 2024-09-16T12:59:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794853+0000", 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3441000000000003e-05 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794913+0000", 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9157999999999998e-05 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:428 4.e e37/31 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head v 37'9, mlcod=37'7)", 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:16.973119+0000", 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.7935470690000002, 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019093409999999999, 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.973119+0000", 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.973119+0000", 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.973120+0000", 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.477e-06 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.973193+0000", 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2995000000000002e-05 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.973222+0000", 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8691e-05 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.973236+0000", 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4127999999999999e-05 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.973317+0000", 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0723999999999995e-05 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.973355+0000", 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8381999999999999e-05 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.974954+0000", 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015984339999999999 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:16.975028+0000", 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4510000000000003e-05 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:482 4.e e37/31 4:71fee990:::rbd_data.11b59d02cc38.00000000000000a5:head v 37'14, mlcod=37'12)", 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.101906+0000", 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.6647602180000001, 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001896874, 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.101906+0000", 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.101906+0000", 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.101907+0000", 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3039999999999999e-06 2024-09-16T12:59:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.102009+0000", 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101777 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.102041+0000", 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2100000000000001e-05 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.102052+0000", 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1589e-05 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.102150+0000", 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8200000000000002e-05 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.102203+0000", 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2573e-05 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.103704+0000", 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001500482 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.103802+0000", 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8849000000000002e-05 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:485 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.6584828, 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066273469999999996, 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 485 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108185+0000", 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3700000000000002e-06 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108195+0000", 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2839999999999995e-06 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108220+0000", 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5066999999999999e-05 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108229+0000", 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5019999999999998e-06 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108334+0000", 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105136 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108571+0000", 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236495 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.109793+0000", 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012221809999999999 2024-09-16T12:59:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114733+0000", 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049399350000000003 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114745+0000", 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2228999999999999e-05 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114783+0000", 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7716999999999998e-05 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114810+0000", 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7430999999999999e-05 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:508 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.180923+0000", 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.585742711, 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036151009999999999, 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 508 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180923+0000", 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180923+0000", 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180926+0000", 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7310000000000002e-06 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180937+0000", 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1559999999999999e-05 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180968+0000", 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1161999999999997e-05 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180981+0000", 2024-09-16T12:59:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.204e-05 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181113+0000", 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131989 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181368+0000", 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000255505 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181780+0000", 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041219100000000001 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183364+0000", 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001584118 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.184372+0000", 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010074120000000001 2024-09-16T12:59:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.184478+0000", 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106127 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.184538+0000", 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0266000000000002e-05 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:510 4.e e37/31 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head v 37'16, mlcod=37'14)", 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.183094+0000", 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.5835717379999998, 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027305430000000002, 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183094+0000", 2024-09-16T12:59:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183094+0000", 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183096+0000", 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.688e-06 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183108+0000", 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.224e-05 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183138+0000", 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0082000000000001e-05 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183153+0000", 2024-09-16T12:59:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5111e-05 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183289+0000", 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013621400000000001 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183351+0000", 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1989999999999994e-05 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.185726+0000", 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002375032 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.185825+0000", 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8185999999999998e-05 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:516 4.e e37/31 4:71fee990:::rbd_data.11b59d02cc38.00000000000000a5:head v 37'17, mlcod=37'14)", 2024-09-16T12:59:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.183958+0000", 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.58270782, 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025765549999999999, 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183958+0000", 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183958+0000", 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183959+0000", 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2950000000000001e-06 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183973+0000", 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3535e-05 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183997+0000", 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4015e-05 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.184011+0000", 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4005999999999999e-05 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.184145+0000", 2024-09-16T12:59:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013391100000000001 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.184191+0000", 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6724000000000002e-05 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186450+0000", 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022582739999999998 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186534+0000", 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4795000000000004e-05 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:590 4.e e37/31 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head v 37'25, mlcod=37'23)", 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.360367+0000", 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4062992160000001, 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001954342, 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.360367+0000", 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.360367+0000", 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.360370+0000", 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1030000000000002e-06 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.360417+0000", 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7555999999999997e-05 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.360460+0000", 2024-09-16T12:59:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2499000000000001e-05 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.360477+0000", 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7509000000000001e-05 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.360606+0000", 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012923700000000001 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.360657+0000", 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0158999999999999e-05 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.362232+0000", 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001574935 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.362321+0000", 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9344000000000004e-05 2024-09-16T12:59:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:635 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2930802990000001, 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046232310000000002, 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 635 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473588+0000", 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9529999999999999e-06 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473602+0000", 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3266e-05 2024-09-16T12:59:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473632+0000", 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0015e-05 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473641+0000", 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5359999999999998e-06 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473734+0000", 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3286000000000001e-05 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473875+0000", 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140211 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.474719+0000", 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084420100000000004 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.477891+0000", 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031721079999999999 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478108+0000", 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021716899999999999 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478167+0000", 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8428999999999999e-05 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478209+0000", 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2057000000000001e-05 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:639 4.5 4:a763b0a4:::rbd_data.11b59d02cc38.0000000000000010:head [set-alloc-hint object_size 1048576 write_size 1048576,write 188238~8370] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.507199+0000", 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2594668439999999, 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026809149999999999, 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 639 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507199+0000", 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507199+0000", 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507202+0000", 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6960000000000001e-06 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507228+0000", 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6747999999999999e-05 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507261+0000", 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2580999999999999e-05 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507277+0000", 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6002999999999999e-05 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507383+0000", 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106561 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507625+0000", 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024115399999999999 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507811+0000", 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018602899999999999 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509258+0000", 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001447313 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509656+0000", 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039810900000000001 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509788+0000", 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132211 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509880+0000", 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1509999999999996e-05 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:640 4.18 e37/31 4:1b76a142:::rbd_data.11b59d02cc38.0000000000000011:head v 37'5, mlcod=36'3)", 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.509652+0000", 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2570137130000001, 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018470380000000001, 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509652+0000", 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509652+0000", 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509653+0000", 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9299999999999997e-07 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509716+0000", 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2749000000000005e-05 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509734+0000", 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8896999999999999e-05 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509746+0000", 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1654999999999999e-05 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509841+0000", 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5105000000000005e-05 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.509901+0000", 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9456000000000001e-05 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.511398+0000", 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014976 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.511499+0000", 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100783 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:748 4.5 4:a3b52eb9:::rbd_data.11b59d02cc38.0000000000000057:head [set-alloc-hint object_size 1048576 write_size 1048576,write 786432~43997] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.774835+0000", 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.99183101699999998, 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031182269999999999, 2024-09-16T12:59:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 748 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774835+0000", 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774835+0000", 2024-09-16T12:59:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774838+0000", 2024-09-16T12:59:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9229999999999998e-06 2024-09-16T12:59:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774993+0000", 2024-09-16T12:59:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015497699999999999 2024-09-16T12:59:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.775031+0000", 2024-09-16T12:59:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8679e-05 2024-09-16T12:59:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.775044+0000", 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2266000000000001e-05 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.775147+0000", 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103006 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.775484+0000", 2024-09-16T12:59:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033703500000000001 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.775815+0000", 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000331233 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.777110+0000", 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00129484 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.777780+0000", 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067046499999999997 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.777877+0000", 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6473999999999998e-05 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.777953+0000", 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6328999999999994e-05 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:847 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [write 929858~25735] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.974876+0000", 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.79179005000000002, 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002952447, 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 847 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974876+0000", 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974876+0000", 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974878+0000", 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4619999999999999e-06 2024-09-16T12:59:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974923+0000", 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4413000000000003e-05 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974950+0000", 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7852e-05 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974960+0000", 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4960000000000002e-06 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.975052+0000", 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2431999999999999e-05 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.975194+0000", 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014174000000000001 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.975378+0000", 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018356199999999999 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.977394+0000", 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020158580000000001 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.977758+0000", 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036432499999999998 2024-09-16T12:59:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.977795+0000", 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7190999999999997e-05 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.977828+0000", 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3115999999999999e-05 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:913 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [write 150823~38808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.116838+0000", 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.64982814499999997, 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002472311, 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 913 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.844 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.116838+0000", 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.116838+0000", 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.116840+0000", 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9520000000000001e-06 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.116912+0000", 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2078999999999999e-05 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.116935+0000", 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2993000000000002e-05 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.116945+0000", 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0732e-05 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.117016+0000", 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0156999999999999e-05 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.117134+0000", 2024-09-16T12:59:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118585 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.117299+0000", 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016523199999999999 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.118586+0000", 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001286602 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.119195+0000", 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060930399999999997 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.119260+0000", 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5062999999999997e-05 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.119310+0000", 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9611999999999999e-05 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:966 4.13 e38/31 4:cb8a2b7c:::rbd_data.11b59d02cc38.00000000000000c2:head v 38'65, mlcod=37'63)", 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.239595+0000", 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.52707047200000001, 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020026509999999998, 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.239595+0000", 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.239595+0000", 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.239597+0000", 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6980000000000001e-06 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.239663+0000", 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5921999999999997e-05 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.239695+0000", 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2209999999999998e-05 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.239708+0000", 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2689e-05 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.239805+0000", 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7147999999999999e-05 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.239854+0000", 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8922999999999999e-05 2024-09-16T12:59:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.241523+0000", 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016694539999999999 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.241598+0000", 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4606999999999999e-05 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:987 4.5 4:a763b0a4:::rbd_data.11b59d02cc38.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.307085+0000", 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.45958122000000001, 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002561486, 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 987 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307085+0000", 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307085+0000", 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307087+0000", 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.756e-06 2024-09-16T12:59:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307099+0000", 2024-09-16T12:59:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1420000000000001e-05 2024-09-16T12:59:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307133+0000", 2024-09-16T12:59:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4415999999999997e-05 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307154+0000", 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1296000000000001e-05 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307235+0000", 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0255999999999999e-05 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307323+0000", 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8636999999999999e-05 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307464+0000", 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014078100000000001 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.308691+0000", 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001226869 2024-09-16T12:59:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.309542+0000", 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085150700000000005 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.309600+0000", 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7686000000000003e-05 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.309646+0000", 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5862000000000002e-05 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:990 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.307224+0000", 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.45944205999999999, 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022818880000000001, 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 990 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307224+0000", 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307224+0000", 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307226+0000", 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.21e-06 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307234+0000", 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5129999999999996e-06 2024-09-16T12:59:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307256+0000", 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.156e-05 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307268+0000", 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2480000000000001e-05 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307341+0000", 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2908000000000001e-05 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307392+0000", 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0451000000000002e-05 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.307461+0000", 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9093999999999999e-05 2024-09-16T12:59:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.308702+0000", 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001241053 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.309254+0000", 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055211699999999997 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.309375+0000", 2024-09-16T12:59:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120457 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.309506+0000", 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013104499999999999 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:988 4.18 e38/31 4:1b76a142:::rbd_data.11b59d02cc38.0000000000000011:head v 38'6, mlcod=36'4)", 2024-09-16T12:59:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.308061+0000", 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.45860501300000001, 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022302350000000001, 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.308061+0000", 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.308061+0000", 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.308061+0000", 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9599999999999999e-07 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.308065+0000", 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1479999999999999e-06 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.308072+0000", 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0390000000000001e-06 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.308077+0000", 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039999999999999e-06 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.308351+0000", 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027402500000000001 2024-09-16T12:59:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.308482+0000", 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130927 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.310256+0000", 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017735349999999999 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.310291+0000", 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5261e-05 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1033 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [set-alloc-hint object_size 1048576 write_size 1048576,write 684157~29432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.342111+0000", 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.42455444799999997, 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001998325, 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1033 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.342111+0000", 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.342111+0000", 2024-09-16T12:59:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.342113+0000", 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8419999999999999e-06 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.342142+0000", 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9226000000000001e-05 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.342168+0000", 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5615000000000001e-05 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.342180+0000", 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1973999999999999e-05 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.342268+0000", 2024-09-16T12:59:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8392000000000003e-05 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.342362+0000", 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3259999999999998e-05 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.342515+0000", 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015361300000000001 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.343547+0000", 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001032207 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.344016+0000", 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046902800000000002 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.344069+0000", 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2308999999999997e-05 2024-09-16T12:59:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.344110+0000", 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0859000000000003e-05 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1035 4.5 4:a763b0a4:::rbd_data.11b59d02cc38.0000000000000010:head [set-alloc-hint object_size 1048576 write_size 1048576,write 244610~14108] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.348006+0000", 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.418659848, 2024-09-16T12:59:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025057550000000001, 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1035 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.348006+0000", 2024-09-16T12:59:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.348006+0000", 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.348008+0000", 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7230000000000001e-06 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.348030+0000", 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1939000000000001e-05 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.348053+0000", 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.368e-05 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.348062+0000", 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9169999999999995e-06 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.348163+0000", 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010073099999999999 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.348318+0000", 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015502800000000001 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.348611+0000", 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029325 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.349965+0000", 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013541709999999999 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.350411+0000", 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000446162 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.350467+0000", 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5739000000000003e-05 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.350512+0000", 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4415000000000001e-05 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:18.885 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:18.885 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:18.885 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T12:59:18.886 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone1 intact 2024-09-16T12:59:18.887 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:18.887 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:18.879+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:18.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:18.966 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-16T12:59:18.959+0000 7f5982d84700 -1 mon.a@0(leader).osd e39 definitely_dead 0 2024-09-16T12:59:18.987 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:18.983+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:19.000 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-16T12:59:19.007 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:19.007 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:19.007 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:19.007 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:19.017 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:19.019 INFO:teuthology.orchestra.run.smithi042.stdout:359 write 0xa047711 thru 0xa055f4b (0xe83b bytes) 2024-09-16T12:59:19.044 INFO:teuthology.orchestra.run.smithi042.stdout:360 trunc from 0xe3c83e7 to 0xd304712 2024-09-16T12:59:19.064 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:19.064 INFO:teuthology.orchestra.run.smithi042.stdout:362 read 0xa760794 thru 0xa76d77f (0xcfec bytes) 2024-09-16T12:59:19.065 INFO:teuthology.orchestra.run.smithi042.stdout:363 trunc from 0xd304712 to 0xc5003f6 2024-09-16T12:59:19.088 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:19.083+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:19.099 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:19.103 INFO:teuthology.orchestra.run.smithi042.stdout:365 clone 3 order 23 su 4194304 sc 11 2024-09-16T12:59:19.119 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:19.120 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:19.120 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:19.120 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:19.120 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:19.121 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:19.130 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:19.136 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:19.155 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:19.175 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:19.188 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:19.183+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:19.198 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:19.225 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:19.248 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:19.250 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:19.275 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:19.275 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:19.284 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:19.289 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:19.283+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:19.390 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:19.387+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:19.391 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:19.392 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:19.402 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:19.491 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:19.487+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:19.502 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:19.502 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:19.511 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:19.592 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:19.587+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:19.613 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:19.613 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:19.623 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:19.694 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:19.687+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:19.720 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:19.720 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:19.729 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:19.796 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:19.791+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:19.833 INFO:teuthology.orchestra.run.smithi042.stderr:marked out osd.1. 2024-09-16T12:59:19.833 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T12:59:19.849 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:19.849 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:19.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:19.903 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:19.899+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:19.953 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:19.954 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:19.963 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:19.997 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:19.991+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:20.063 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:20.063 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:20.072 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:20.098 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:20.091+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:20.176 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:20.176 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:20.185 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:20.199 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:20.195+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:20.287 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:20.287 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:20.296 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:20.299 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:20.295+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:20.398 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:20.398 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:20.399 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:20.395+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:20.408 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:20.500 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:20.495+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:20.502 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:20.502 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:20.512 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:20.602 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:20.595+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:20.608 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:20.608 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:20.617 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:20.703 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:20.699+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:20.726 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:20.726 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:20.735 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:20.804 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:20.799+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:20.839 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:20.839 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:20.850 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:20.904 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:20.899+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:20.946 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:20.946 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:20.956 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:20.972 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone2 2024-09-16T12:59:21.005 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:20.999+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:21.106 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:21.095+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:21.113 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:21.113 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:21.123 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:21.207 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:21.203+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:21.220 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:21.221 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:21.230 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:21.262 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:21.307 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:21.303+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:21.331 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:21.332 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:21.340 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:21.408 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:21.399+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:21.427 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:21.437 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:21.437 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:21.446 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:21.447 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:21.467 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:21.490 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:21.509 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:21.499+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:21.512 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-16T12:59:21.512 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:21.534 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:21.544 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:21.544 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:21.553 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:21.556 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:21.579 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:21.610 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:21.607+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:21.656 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 7.8663708010000004, 2024-09-16T12:59:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089374989999999998, 2024-09-16T12:59:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 11 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785978+0000", 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.373e-06 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785989+0000", 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1596999999999999e-05 2024-09-16T12:59:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786008+0000", 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8510999999999999e-05 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786016+0000", 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786102+0000", 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6339000000000001e-05 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786113+0000", 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1386000000000001e-05 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.788774+0000", 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026609519999999998 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.788888+0000", 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113189 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789043+0000", 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015570800000000001 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794705+0000", 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056619640000000002 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794770+0000", 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4789999999999995e-05 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794853+0000", 2024-09-16T12:59:21.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3441000000000003e-05 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794913+0000", 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9157999999999998e-05 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.663 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:485 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5441630159999997, 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066273469999999996, 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 485 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108185+0000", 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3700000000000002e-06 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108195+0000", 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2839999999999995e-06 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108220+0000", 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5066999999999999e-05 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108229+0000", 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5019999999999998e-06 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108334+0000", 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105136 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108571+0000", 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236495 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.109793+0000", 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012221809999999999 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114733+0000", 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049399350000000003 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114745+0000", 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2228999999999999e-05 2024-09-16T12:59:21.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114783+0000", 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7716999999999998e-05 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114810+0000", 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7430999999999999e-05 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.669 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:508 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.180923+0000", 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4714229269999999, 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036151009999999999, 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 508 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180923+0000", 2024-09-16T12:59:21.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180923+0000", 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180926+0000", 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7310000000000002e-06 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180937+0000", 2024-09-16T12:59:21.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1559999999999999e-05 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180968+0000", 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1161999999999997e-05 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.180981+0000", 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.204e-05 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181113+0000", 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131989 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181368+0000", 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000255505 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181780+0000", 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041219100000000001 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.183364+0000", 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001584118 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.184372+0000", 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010074120000000001 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.184478+0000", 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106127 2024-09-16T12:59:21.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.184538+0000", 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0266000000000002e-05 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:635 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:21.675 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.1787605150000005, 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046232310000000002, 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 635 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:21.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:21.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473588+0000", 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9529999999999999e-06 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473602+0000", 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3266e-05 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473632+0000", 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0015e-05 2024-09-16T12:59:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473641+0000", 2024-09-16T12:59:21.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5359999999999998e-06 2024-09-16T12:59:21.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473734+0000", 2024-09-16T12:59:21.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3286000000000001e-05 2024-09-16T12:59:21.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473875+0000", 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140211 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.474719+0000", 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084420100000000004 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.477891+0000", 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031721079999999999 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478108+0000", 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021716899999999999 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478167+0000", 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8428999999999999e-05 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478209+0000", 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2057000000000001e-05 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:748 4.5 4:a3b52eb9:::rbd_data.11b59d02cc38.0000000000000057:head [set-alloc-hint object_size 1048576 write_size 1048576,write 786432~43997] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.774835+0000", 2024-09-16T12:59:21.682 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8775112329999999, 2024-09-16T12:59:21.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031182269999999999, 2024-09-16T12:59:21.683 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.683 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.683 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:21.683 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:21.683 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:21.683 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 748 2024-09-16T12:59:21.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.683 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774835+0000", 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774835+0000", 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774838+0000", 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9229999999999998e-06 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774993+0000", 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015497699999999999 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.775031+0000", 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8679e-05 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.775044+0000", 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2266000000000001e-05 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.775147+0000", 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103006 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.775484+0000", 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033703500000000001 2024-09-16T12:59:21.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.775815+0000", 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000331233 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.777110+0000", 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00129484 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.777780+0000", 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067046499999999997 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.777877+0000", 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6473999999999998e-05 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.777953+0000", 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6328999999999994e-05 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:847 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [write 929858~25735] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.974876+0000", 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.6774702660000003, 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002952447, 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 847 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974876+0000", 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974876+0000", 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974878+0000", 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4619999999999999e-06 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974923+0000", 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4413000000000003e-05 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974950+0000", 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7852e-05 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.974960+0000", 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4960000000000002e-06 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.975052+0000", 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2431999999999999e-05 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.975194+0000", 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014174000000000001 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.975378+0000", 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018356199999999999 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.977394+0000", 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020158580000000001 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.977758+0000", 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036432499999999998 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.977795+0000", 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7190999999999997e-05 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.977828+0000", 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3115999999999999e-05 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1247 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'10, mlcod=39'8)", 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.027164+0000", 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.6251814590000002, 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038273140000000001, 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027164+0000", 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027164+0000", 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027167+0000", 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2840000000000001e-06 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027184+0000", 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6816e-05 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027218+0000", 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4289000000000003e-05 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027234+0000", 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5616000000000001e-05 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027303+0000", 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9214000000000005e-05 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027325+0000", 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2656000000000001e-05 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030901+0000", 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035752679999999999 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030992+0000", 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1170999999999998e-05 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1248 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'11, mlcod=39'8)", 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.624303931, 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053616649999999998, 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028044+0000", 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5e-06 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028058+0000", 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4469e-05 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028091+0000", 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2517000000000001e-05 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028106+0000", 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5116000000000001e-05 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028219+0000", 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113768 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028268+0000", 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8745000000000002e-05 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033313+0000", 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050452869999999999 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033404+0000", 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0263e-05 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1249 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'12, mlcod=39'8)", 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.029057+0000", 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.6232888029999999, 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044055769999999999, 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029057+0000", 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029057+0000", 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029059+0000", 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5370000000000001e-06 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029070+0000", 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1593000000000001e-05 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029095+0000", 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4505999999999999e-05 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029105+0000", 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7769999999999997e-06 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029214+0000", 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109887 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029273+0000", 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8709000000000003e-05 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033408+0000", 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041346880000000001 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033463+0000", 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4880000000000003e-05 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1250 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'13, mlcod=39'8)", 2024-09-16T12:59:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.6219828509999998, 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061776180000000002, 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030364+0000", 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.356e-06 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030375+0000", 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0053e-05 2024-09-16T12:59:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030395+0000", 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0767e-05 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030404+0000", 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4430000000000003e-06 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030492+0000", 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7842000000000004e-05 2024-09-16T12:59:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030514+0000", 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2819999999999998e-05 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036453+0000", 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059381169999999997 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036541+0000", 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8220000000000006e-05 2024-09-16T12:59:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1251 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'14, mlcod=39'8)", 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.032254+0000", 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.6200920989999998, 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043426029999999996, 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032254+0000", 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032254+0000", 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032255+0000", 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1000000000000001e-06 2024-09-16T12:59:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032263+0000", 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5660000000000001e-06 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032282+0000", 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9184000000000001e-05 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032290+0000", 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6340000000000007e-06 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032330+0000", 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9375000000000002e-05 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032349+0000", 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9612000000000001e-05 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036546+0000", 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041971630000000003 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036596+0000", 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9969000000000002e-05 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1252 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'15, mlcod=39'9)", 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.033997+0000", 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.6183485819999999, 2024-09-16T12:59:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047268359999999999, 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033997+0000", 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033997+0000", 2024-09-16T12:59:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033998+0000", 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.054e-06 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034023+0000", 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5041e-05 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034048+0000", 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4097999999999999e-05 2024-09-16T12:59:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034058+0000", 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0515e-05 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034128+0000", 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0362000000000002e-05 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034165+0000", 2024-09-16T12:59:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6600999999999997e-05 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.038652+0000", 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044872999999999996 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.038724+0000", 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1865000000000004e-05 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1253 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'16, mlcod=39'11)", 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.035656+0000", 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.6166896340000001, 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038991009999999999, 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035656+0000", 2024-09-16T12:59:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035656+0000", 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035657+0000", 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0100000000000001e-06 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035665+0000", 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1499999999999999e-06 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035682+0000", 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6351e-05 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035689+0000", 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0539999999999997e-06 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035750+0000", 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1097000000000001e-05 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035785+0000", 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4773000000000001e-05 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.039498+0000", 2024-09-16T12:59:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037130800000000001 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.039555+0000", 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7586000000000001e-05 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1263 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'19, mlcod=39'15)", 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.047178+0000", 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.6051680560000001, 2024-09-16T12:59:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045177070000000001, 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047178+0000", 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047178+0000", 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047179+0000", 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000003e-07 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047197+0000", 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8629000000000001e-05 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047211+0000", 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3699999999999999e-05 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047218+0000", 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1040000000000001e-06 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047273+0000", 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4483e-05 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047297+0000", 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.438e-05 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051620+0000", 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043228920000000001 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051696+0000", 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5604000000000003e-05 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1264 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'20, mlcod=39'15)", 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.047902+0000", 2024-09-16T12:59:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.6044439329999998, 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038547030000000001, 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047902+0000", 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047902+0000", 2024-09-16T12:59:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047903+0000", 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7199999999999998e-07 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047909+0000", 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1299999999999998e-06 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047922+0000", 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2744e-05 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047927+0000", 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4489999999999998e-06 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047972+0000", 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5275000000000003e-05 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047998+0000", 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5381999999999999e-05 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051699+0000", 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037010509999999999 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051757+0000", 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7899999999999998e-05 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1265 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'21, mlcod=39'16)", 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.048976+0000", 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.6033699320000001, 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034550269999999998, 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.048976+0000", 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.048976+0000", 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.048977+0000", 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8199999999999999e-07 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.048982+0000", 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7819999999999999e-06 2024-09-16T12:59:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.048998+0000", 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5560999999999999e-05 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.049005+0000", 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7700000000000004e-06 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.049047+0000", 2024-09-16T12:59:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2361999999999999e-05 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.049068+0000", 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.065e-05 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.052395+0000", 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033268059999999999 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.052431+0000", 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6414000000000001e-05 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1296 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'29, mlcod=39'25)", 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.573209324, 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004900616, 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1800000000000005e-07 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079145+0000", 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3760000000000002e-06 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079159+0000", 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4438e-05 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079168+0000", 2024-09-16T12:59:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2299999999999997e-06 2024-09-16T12:59:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079253+0000", 2024-09-16T12:59:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4686999999999997e-05 2024-09-16T12:59:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079299+0000", 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6072000000000002e-05 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084001+0000", 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047020100000000004 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084037+0000", 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5985000000000002e-05 2024-09-16T12:59:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1301 4.5 4:a692e91d:::rbd_data.11b53140b009.00000000000000c4:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.5719456570000001, 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060951929999999996, 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1301 2024-09-16T12:59:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080402+0000", 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.75e-06 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080409+0000", 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2219999999999996e-06 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080429+0000", 2024-09-16T12:59:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9527000000000001e-05 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080438+0000", 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7819999999999999e-06 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080503+0000", 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5438999999999995e-05 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080693+0000", 2024-09-16T12:59:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190218 2024-09-16T12:59:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080909+0000", 2024-09-16T12:59:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021529000000000001 2024-09-16T12:59:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084016+0000", 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031074319999999998 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086352+0000", 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023357690000000001 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086446+0000", 2024-09-16T12:59:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4158000000000002e-05 2024-09-16T12:59:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086495+0000", 2024-09-16T12:59:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9605999999999999e-05 2024-09-16T12:59:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1297 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'30, mlcod=39'27)", 2024-09-16T12:59:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.081247+0000", 2024-09-16T12:59:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.5710989930000001, 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050937860000000003, 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081247+0000", 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081247+0000", 2024-09-16T12:59:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081248+0000", 2024-09-16T12:59:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7199999999999997e-07 2024-09-16T12:59:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081255+0000", 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2169999999999997e-06 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081272+0000", 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6671e-05 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081278+0000", 2024-09-16T12:59:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4350000000000002e-06 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081341+0000", 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3001000000000002e-05 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081365+0000", 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3247e-05 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086290+0000", 2024-09-16T12:59:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049256860000000003 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086341+0000", 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0556999999999997e-05 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1298 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'31, mlcod=39'27)", 2024-09-16T12:59:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.083232+0000", 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.5691143040000002, 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039085630000000003, 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083232+0000", 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083232+0000", 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083233+0000", 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0020000000000001e-06 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083243+0000", 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0441999999999999e-05 2024-09-16T12:59:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083267+0000", 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3706999999999999e-05 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083276+0000", 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4239999999999999e-06 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083345+0000", 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8571999999999993e-05 2024-09-16T12:59:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083367+0000", 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2405999999999999e-05 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.087081+0000", 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037133299999999999 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.087140+0000", 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9679999999999998e-05 2024-09-16T12:59:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:21.760 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:21.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:21.762 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:21.755+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:21.844 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T12:59:21.869 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:21.869 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:21.869 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:21.869 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:21.878 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:21.891 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:21.883+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:21.909 INFO:teuthology.orchestra.run.smithi042.stdout:366 punch from 0x959dc4b to 0x959eaeb, (0xea0 bytes) 2024-09-16T12:59:21.916 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:21.916 INFO:teuthology.orchestra.run.smithi042.stdout:368 write 0xbd61bb6 thru 0xbd6d4f2 (0xb93d bytes) 2024-09-16T12:59:21.922 INFO:teuthology.orchestra.run.smithi042.stdout:369 read 0x6931ef4 thru 0x69401b0 (0xe2bd bytes) 2024-09-16T12:59:21.922 INFO:teuthology.orchestra.run.smithi042.stdout:370 read 0xb4a6ae3 thru 0xb4b6325 (0xf843 bytes) 2024-09-16T12:59:21.923 INFO:teuthology.orchestra.run.smithi042.stdout:371 write 0x9b82af9 thru 0x9b8bde6 (0x92ee bytes) 2024-09-16T12:59:21.930 INFO:teuthology.orchestra.run.smithi042.stdout:372 punch from 0x9a79a25 to 0x9a7fbeb, (0x61c6 bytes) 2024-09-16T12:59:21.930 INFO:teuthology.orchestra.run.smithi042.stdout:375 write 0x82dd90d thru 0x82e2c9c (0x5390 bytes) 2024-09-16T12:59:21.939 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:21.939 INFO:teuthology.orchestra.run.smithi042.stdout:377 read 0x4070e64 thru 0x407773c (0x68d9 bytes) 2024-09-16T12:59:21.953 INFO:teuthology.orchestra.run.smithi042.stdout:379 write 0xe520f04 thru 0xe521409 (0x506 bytes) 2024-09-16T12:59:21.960 INFO:teuthology.orchestra.run.smithi042.stdout:380 writesame 0xa793238 thru 0xa79ef1f data_size 0x82(0xbce8 bytes) 2024-09-16T12:59:21.963 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:21.959+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:21.979 INFO:teuthology.orchestra.run.smithi042.stdout:383 write 0x834117c thru 0x834df42 (0xcdc7 bytes) 2024-09-16T12:59:21.980 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:21.980 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:21.980 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:21.980 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:21.980 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:21.981 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:21.982 INFO:teuthology.orchestra.run.smithi042.stdout:385 read 0x8a6c5dd thru 0x8a70d25 (0x4749 bytes) 2024-09-16T12:59:21.982 INFO:teuthology.orchestra.run.smithi042.stdout:386 read 0xa7911ae thru 0xa798285 (0x70d8 bytes) 2024-09-16T12:59:21.983 INFO:teuthology.orchestra.run.smithi042.stdout:387 read 0xafae617 thru 0xafb97ff (0xb1e9 bytes) 2024-09-16T12:59:21.989 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:22.008 INFO:teuthology.orchestra.run.smithi042.stdout:391 trunc from 0xe52140a to 0xe62946 2024-09-16T12:59:22.055 INFO:teuthology.orchestra.run.smithi042.stdout:392 writesame 0x280982e thru 0x2817c5e data_size 0x1(0xe431 bytes) 2024-09-16T12:59:22.064 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:22.055+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:22.067 INFO:teuthology.orchestra.run.smithi042.stdout:393 write 0x3d8f4e7 thru 0x3d98061 (0x8b7b bytes) 2024-09-16T12:59:22.073 INFO:teuthology.orchestra.run.smithi042.stdout:395 read 0x2f02e5 thru 0x2fea43 (0xe75f bytes) 2024-09-16T12:59:22.074 INFO:teuthology.orchestra.run.smithi042.stdout:397 read 0x20a1ee4 thru 0x20aeecb (0xcfe8 bytes) 2024-09-16T12:59:22.089 INFO:teuthology.orchestra.run.smithi042.stdout:398 trunc from 0x3d98062 to 0xc7c823a 2024-09-16T12:59:22.095 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:22.095 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:22.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:22.095 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:22.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.095 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:22.095 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:22.095 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.539711093000001, 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:22.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T12:59:22.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.714884509000001, 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010809278, 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.102 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374549+0000", 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.037e-06 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374711+0000", 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016226599999999999 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374734+0000", 2024-09-16T12:59:22.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2895999999999999e-05 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374746+0000", 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.221e-05 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.380942+0000", 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061959390000000001 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381063+0000", 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121274 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381255+0000", 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019110500000000001 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384487+0000", 2024-09-16T12:59:22.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032327979999999998 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385296+0000", 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080888499999999996 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385332+0000", 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5774e-05 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385357+0000", 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5094000000000001e-05 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.714673617000003, 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010976025, 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.108 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 45 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374760+0000", 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.13e-06 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374769+0000", 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5699999999999999e-06 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374819+0000", 2024-09-16T12:59:22.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9882000000000001e-05 2024-09-16T12:59:22.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374828+0000", 2024-09-16T12:59:22.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-16T12:59:22.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382329+0000", 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075009009999999999 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382403+0000", 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4208999999999994e-05 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382514+0000", 2024-09-16T12:59:22.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111276 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385533+0000", 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030190080000000001 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385564+0000", 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0926000000000002e-05 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385666+0000", 2024-09-16T12:59:22.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101791 2024-09-16T12:59:22.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385735+0000", 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8541999999999995e-05 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.714594046000002, 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:22.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T12:59:22.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T12:59:22.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.71454679, 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T12:59:22.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T12:59:22.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.126 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.714343853999999, 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T12:59:22.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.132 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.714312591999999, 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.138 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.714276830000003, 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T12:59:22.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T12:59:22.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T12:59:22.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T12:59:22.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T12:59:22.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.144 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.7140822, 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.146 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T12:59:22.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.146 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:22.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T12:59:22.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T12:59:22.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T12:59:22.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T12:59:22.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T12:59:22.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.714035355, 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.151 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:22.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T12:59:22.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T12:59:22.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T12:59:22.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T12:59:22.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T12:59:22.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T12:59:22.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T12:59:22.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.169975313999998, 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010623719, 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 69 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919458+0000", 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.474e-06 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919621+0000", 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016234800000000001 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919641+0000", 2024-09-16T12:59:22.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0336999999999999e-05 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919656+0000", 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4554e-05 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924273+0000", 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046169610000000002 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924411+0000", 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138286 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924627+0000", 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021597399999999999 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928794+0000", 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004167268 2024-09-16T12:59:22.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929983+0000", 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011884409999999999 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930039+0000", 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6363000000000002e-05 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930081+0000", 2024-09-16T12:59:22.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1712999999999999e-05 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.169759184, 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.164 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T12:59:22.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T12:59:22.169 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.169705276999998, 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.170 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T12:59:22.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.175 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.169662971999998, 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T12:59:22.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T12:59:22.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T12:59:22.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T12:59:22.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T12:59:22.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.169504547000003, 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T12:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.169455065999998, 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.169324496000002, 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.194 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T12:59:22.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T12:59:22.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T12:59:22.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T12:59:22.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T12:59:22.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T12:59:22.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T12:59:22.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T12:59:22.200 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.200 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.200 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.200 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:22.200 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:22.200 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.169142983, 2024-09-16T12:59:22.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T12:59:22.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T12:59:22.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.358615315999998, 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.207 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T12:59:22.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T12:59:22.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T12:59:22.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T12:59:22.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T12:59:22.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T12:59:22.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.358466362000001, 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T12:59:22.213 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T12:59:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:22.219 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:22.220 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:22.222 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:22.215+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:22.222 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:22.222 INFO:teuthology.orchestra.run.smithi042.stdout:399 writesame 0x7f5e95d thru 0x7f69f99 data_size 0x1(0xb63d bytes) 2024-09-16T12:59:22.223 INFO:teuthology.orchestra.run.smithi042.stdout:400 write 0xccc00ca thru 0xccc7c26 (0x7b5d bytes) 2024-09-16T12:59:22.223 INFO:teuthology.orchestra.run.smithi042.stdout:401 read 0x3490d61 thru 0x349a7ca (0x9a6a bytes) 2024-09-16T12:59:22.223 INFO:teuthology.orchestra.run.smithi042.stdout:402 trunc from 0xccc7c27 to 0x7314aa3 2024-09-16T12:59:22.223 INFO:teuthology.orchestra.run.smithi042.stdout:403 punch from 0xf1b864 to 0xf2b2c5, (0xfa61 bytes) 2024-09-16T12:59:22.223 INFO:teuthology.orchestra.run.smithi042.stdout:404 write 0x30e1915 thru 0x30e6250 (0x493c bytes) 2024-09-16T12:59:22.223 INFO:teuthology.orchestra.run.smithi042.stdout:405 trunc from 0x7314aa3 to 0x59399a2 2024-09-16T12:59:22.223 INFO:teuthology.orchestra.run.smithi042.stdout:406 trunc from 0x59399a2 to 0x1c77046 2024-09-16T12:59:22.227 INFO:teuthology.orchestra.run.smithi042.stdout:408 read 0x1640120 thru 0x1645b44 (0x5a25 bytes) 2024-09-16T12:59:22.260 INFO:teuthology.orchestra.run.smithi042.stdout:411 write 0xc395ac6 thru 0xc39dc29 (0x8164 bytes) 2024-09-16T12:59:22.278 INFO:teuthology.orchestra.run.smithi042.stdout:413 writesame 0xc570282 thru 0xc576549 data_size 0x1d(0x62c8 bytes) 2024-09-16T12:59:22.278 INFO:teuthology.orchestra.run.smithi042.stdout:414 write 0x6088778 thru 0x609499e (0xc227 bytes) 2024-09-16T12:59:22.285 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:22.285 INFO:teuthology.orchestra.run.smithi042.stdout:416 punch from 0x749be52 to 0x74a439f, (0x854d bytes) 2024-09-16T12:59:22.285 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:22.285 INFO:teuthology.orchestra.run.smithi042.stdout:419 read 0x492cc85 thru 0x49359b8 (0x8d34 bytes) 2024-09-16T12:59:22.305 INFO:teuthology.orchestra.run.smithi042.stdout:420 trunc from 0xc57654a to 0x112bbc0 2024-09-16T12:59:22.322 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:22.311+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:22.329 INFO:teuthology.orchestra.run.smithi042.stdout:421 clone 4 order 22 su 1048576 sc 7 2024-09-16T12:59:22.336 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:22.336 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:22.336 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:22.336 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:22.346 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:22.423 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:22.415+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:22.488 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:22.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:22.488 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:22.489 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:22.489 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:22.497 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:22.524 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:22.519+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:22.604 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:22.604 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:22.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:22.604 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:22.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.604 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4210.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-16T12:59:22.604 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:58:46.169566+0000", 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.426883568000001, 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033952489999999999, 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169566+0000", 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169566+0000", 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169568+0000", 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8959999999999999e-06 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169803+0000", 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023505499999999999 2024-09-16T12:59:22.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169836+0000", 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3283e-05 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169852+0000", 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5812999999999999e-05 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.169990+0000", 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138613 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.170035+0000", 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4589000000000003e-05 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.172855+0000", 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028203490000000002 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:58:46.172961+0000", 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105651 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:313 4.1e 4:7abe541a:::rbd_object_map.11b524d5182b.0000000000000004:head [writefull 0~35] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7921217609999998, 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053862449999999996, 2024-09-16T12:59:22.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 313 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804330+0000", 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3309999999999998e-06 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804341+0000", 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1284000000000001e-05 2024-09-16T12:59:22.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804367+0000", 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6132999999999999e-05 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804380+0000", 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2686e-05 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804463+0000", 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3187000000000002e-05 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804472+0000", 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.149e-06 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.806749+0000", 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022770669999999998 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.806912+0000", 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016236500000000001 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.807071+0000", 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000158956 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.808001+0000", 2024-09-16T12:59:22.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093026199999999995 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809575+0000", 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015741609999999999 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809654+0000", 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9028000000000003e-05 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809714+0000", 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9636000000000003e-05 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:440 4.16 4:6ccd461c:::rbd_data.11b59d02cc38.00000000000000d2:head [set-alloc-hint object_size 1048576 write_size 1048576,write 18721~46815] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.007001+0000", 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.5894486140000001, 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032802619999999999, 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 440 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007001+0000", 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007001+0000", 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007003+0000", 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4650000000000001e-06 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007031+0000", 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7798999999999999e-05 2024-09-16T12:59:22.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007060+0000", 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9079000000000001e-05 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007071+0000", 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1358999999999999e-05 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007171+0000", 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9130999999999997e-05 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007398+0000", 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022744599999999999 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.007626+0000", 2024-09-16T12:59:22.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022826199999999999 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.009976+0000", 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023497100000000001 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.010180+0000", 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020415400000000001 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.010234+0000", 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3745000000000001e-05 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.010281+0000", 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7111999999999999e-05 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:486 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:22.622 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4901378950000002, 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077714180000000004, 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 486 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:22.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106314+0000", 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8329999999999998e-06 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106329+0000", 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4231e-05 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106369+0000", 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0074999999999998e-05 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106383+0000", 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4781e-05 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106471+0000", 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7472999999999994e-05 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106667+0000", 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000196503 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.107835+0000", 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011676670000000001 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.113820+0000", 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0059854069999999999 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.113900+0000", 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0071e-05 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.114010+0000", 2024-09-16T12:59:22.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109399 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.114083+0000", 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2978000000000005e-05 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:509 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.178270+0000", 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4181798309999998, 2024-09-16T12:59:22.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030314669999999999, 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 509 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178270+0000", 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178270+0000", 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178271+0000", 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.889e-06 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178281+0000", 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7079999999999992e-06 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178303+0000", 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2007000000000001e-05 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178312+0000", 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4270000000000008e-06 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178365+0000", 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3615000000000001e-05 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178470+0000", 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104678 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178611+0000", 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000141574 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.180072+0000", 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014604189999999999 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.181038+0000", 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096601599999999999 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.181164+0000", 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012594 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.181301+0000", 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013719399999999999 2024-09-16T12:59:22.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:636 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1242503089999998, 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027791489999999999, 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 636 2024-09-16T12:59:22.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472202+0000", 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.01e-06 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472218+0000", 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5724000000000001e-05 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472261+0000", 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2858999999999997e-05 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472277+0000", 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6297999999999999e-05 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472397+0000", 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012026199999999999 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472563+0000", 2024-09-16T12:59:22.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016614900000000001 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.480681+0000", 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081177869999999996 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499671+0000", 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018990289 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499756+0000", 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4511000000000005e-05 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499858+0000", 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102495 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499991+0000", 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132106 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.639 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:677 4.b 4:d203d4b6:::rbd_data.11b59d02cc38.0000000000000013:head [set-alloc-hint object_size 1048576 write_size 1048576,write 800794~21106] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.599747+0000", 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9967028149999999, 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003663641, 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 677 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.599747+0000", 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.599747+0000", 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.599750+0000", 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1949999999999999e-06 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.599782+0000", 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2292999999999997e-05 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.599822+0000", 2024-09-16T12:59:22.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9505000000000002e-05 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.599837+0000", 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5353e-05 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.599948+0000", 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111166 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.600212+0000", 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026435 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.600458+0000", 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000245706 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.602212+0000", 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017540780000000001 2024-09-16T12:59:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.603224+0000", 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00101161 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.603303+0000", 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8894999999999996e-05 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.603410+0000", 2024-09-16T12:59:22.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010749 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:680 4.b 4:d0725c9e:::rbd_data.11b59d02cc38.0000000000000092:head [set-alloc-hint object_size 1048576 write_size 1048576,write 131072~26521] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.607728+0000", 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9887215930000002, 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003676693, 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 680 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.607728+0000", 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.607728+0000", 2024-09-16T12:59:22.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.607730+0000", 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8840000000000001e-06 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.607758+0000", 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8385e-05 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.607786+0000", 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7501e-05 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.607798+0000", 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2189e-05 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.607903+0000", 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105547 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608135+0000", 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023169799999999999 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608422+0000", 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028743099999999999 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.610317+0000", 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001894352 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611212+0000", 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00089496599999999997 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611298+0000", 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5848999999999997e-05 2024-09-16T12:59:22.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611404+0000", 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106891 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:684 4.16 4:6824299a:::rbd_data.11b59d02cc38.0000000000000030:head [set-alloc-hint object_size 1048576 write_size 1048576,write 261969~175] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.616240+0000", 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9802091409999996, 2024-09-16T12:59:22.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033821580000000001, 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 684 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.616240+0000", 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.616240+0000", 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.616242+0000", 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.874e-06 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.616253+0000", 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0535e-05 2024-09-16T12:59:22.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.616284+0000", 2024-09-16T12:59:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1225e-05 2024-09-16T12:59:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.616295+0000", 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1477e-05 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.616415+0000", 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119811 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.616651+0000", 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023587400000000001 2024-09-16T12:59:22.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.616864+0000", 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021307699999999999 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.618573+0000", 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017087370000000001 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.619397+0000", 2024-09-16T12:59:22.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00082426099999999996 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.619522+0000", 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125003 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.619622+0000", 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100284 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:846 4.b 4:d203d4b6:::rbd_data.11b59d02cc38.0000000000000013:head [read 655360~35183] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.967702+0000", 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6287470949999996, 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038915310000000002, 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 846 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.659 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967702+0000", 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967702+0000", 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967705+0000", 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1659999999999998e-06 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967719+0000", 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-05 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967753+0000", 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4866999999999997e-05 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967766+0000", 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.272e-05 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967870+0000", 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010409800000000001 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967880+0000", 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9559999999999992e-06 2024-09-16T12:59:22.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.971326+0000", 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034458790000000002 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.971432+0000", 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010626800000000001 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.971594+0000", 2024-09-16T12:59:22.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016137700000000001 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:989 4.b 4:d203d4b6:::rbd_data.11b59d02cc38.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.304473+0000", 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.291976719, 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038852840000000001, 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.664 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 989 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304473+0000", 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304473+0000", 2024-09-16T12:59:22.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304475+0000", 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7539999999999998e-06 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304484+0000", 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8380000000000008e-06 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304510+0000", 2024-09-16T12:59:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5820999999999999e-05 2024-09-16T12:59:22.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304520+0000", 2024-09-16T12:59:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7010000000000008e-06 2024-09-16T12:59:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304641+0000", 2024-09-16T12:59:22.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120951 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304741+0000", 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010043300000000001 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304869+0000", 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012801699999999999 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.306507+0000", 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001637552 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308257+0000", 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017501179999999999 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308313+0000", 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6060999999999997e-05 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308358+0000", 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5037999999999998e-05 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:991 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.304546+0000", 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2919031240000001, 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003191583, 2024-09-16T12:59:22.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 991 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304546+0000", 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304546+0000", 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304547+0000", 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7599999999999997e-07 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304550+0000", 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2799999999999999e-06 2024-09-16T12:59:22.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304558+0000", 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5739999999999998e-06 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304565+0000", 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.305233+0000", 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00066788600000000002 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.305304+0000", 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0970999999999995e-05 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.305686+0000", 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038187200000000001 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.306846+0000", 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011601389999999999 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307639+0000", 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00079288399999999999 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307691+0000", 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2923000000000001e-05 2024-09-16T12:59:22.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307738+0000", 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6341000000000002e-05 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1078 4.6 4:60e6d7cb:::rbd_data.11b59d02cc38.0000000000000078:head [set-alloc-hint object_size 1048576 write_size 1048576,write 786432~30933] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.435924+0000", 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.160525518, 2024-09-16T12:59:22.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003180113, 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1078 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.435924+0000", 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.435924+0000", 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.435927+0000", 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0299999999999998e-06 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.435983+0000", 2024-09-16T12:59:22.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5878000000000003e-05 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.436015+0000", 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1906000000000003e-05 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.436027+0000", 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2808e-05 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.436133+0000", 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010592800000000001 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.436325+0000", 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000191447 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.436486+0000", 2024-09-16T12:59:22.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016111200000000001 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.438010+0000", 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015242299999999999 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.438995+0000", 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098496799999999991 2024-09-16T12:59:22.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.439051+0000", 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6189e-05 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.439104+0000", 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2617000000000001e-05 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1081 4.3 4:c0127367:::rbd_data.11b59d02cc38.00000000000000ac:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.443370+0000", 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1530797379999997, 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043702059999999997, 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443370+0000", 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443370+0000", 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443372+0000", 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1459999999999999e-06 2024-09-16T12:59:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443394+0000", 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2410000000000001e-05 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443422+0000", 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7464e-05 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443452+0000", 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0241000000000001e-05 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443558+0000", 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644700000000001 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443837+0000", 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027904500000000001 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.444488+0000", 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065053099999999998 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.447187+0000", 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026991390000000001 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.447603+0000", 2024-09-16T12:59:22.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041617100000000001 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.447679+0000", 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5927000000000007e-05 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.447740+0000", 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0684999999999999e-05 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1143 4.1e 4:7abe541a:::rbd_object_map.11b524d5182b.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.729080+0000", 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.8673689630000001, 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029758470000000002, 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1143 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.729080+0000", 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.729080+0000", 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.729085+0000", 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2300000000000002e-06 2024-09-16T12:59:22.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.729105+0000", 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9947000000000001e-05 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.729160+0000", 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.516e-05 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.729180+0000", 2024-09-16T12:59:22.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0724e-05 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.729273+0000", 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3061999999999998e-05 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.729279+0000", 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3229999999999997e-06 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.731822+0000", 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002543399 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.731923+0000", 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100795 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.732056+0000", 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013320699999999999 2024-09-16T12:59:22.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1272 4.1e 4:7ee7bf61:::rbd_data.11b53140b009.00000000000000d5:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.049952+0000", 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5464975270000001, 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002916484, 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1272 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049952+0000", 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049952+0000", 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049954+0000", 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.221e-06 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049964+0000", 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9119999999999993e-06 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049990+0000", 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6543999999999999e-05 2024-09-16T12:59:22.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.050000+0000", 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2080000000000006e-06 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.050076+0000", 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6376000000000002e-05 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.050298+0000", 2024-09-16T12:59:22.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022152599999999999 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.050591+0000", 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029366499999999997 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.051842+0000", 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001251027 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052768+0000", 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00092592399999999995 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052829+0000", 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1154000000000004e-05 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052868+0000", 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8927000000000002e-05 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1292 4.6 4:603e19dc:::rbd_data.11b53140b009.00000000000000d8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.064908+0000", 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5315415679999997, 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029213080000000001, 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1292 2024-09-16T12:59:22.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064908+0000", 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064908+0000", 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064909+0000", 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0299999999999997e-07 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064915+0000", 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6499999999999999e-06 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064925+0000", 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7650000000000005e-06 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064930+0000", 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3059999999999997e-06 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.065924+0000", 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099383599999999989 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.066000+0000", 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5545999999999998e-05 2024-09-16T12:59:22.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.066103+0000", 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103544 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.066987+0000", 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088413200000000004 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.067766+0000", 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000778376 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.067802+0000", 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5871999999999998e-05 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.067829+0000", 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7378000000000001e-05 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3577 4.1e 4:7a8a5a14:::rbd_data.11b5c378985b.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.49580385700000001, 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027798529999999998, 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3577 2024-09-16T12:59:22.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100649+0000", 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1769999999999998e-06 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100659+0000", 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0407e-05 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100689+0000", 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0290999999999998e-05 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100701+0000", 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1960999999999999e-05 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100800+0000", 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8350000000000005e-05 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.101027+0000", 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000227736 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.111830+0000", 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010802735000000001 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128285+0000", 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016454389999999999 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128338+0000", 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3112000000000002e-05 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128398+0000", 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0136999999999997e-05 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128444+0000", 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6233999999999998e-05 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:3577 4.1e e41/31 ondisk, result = 0)", 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.47663782100000002, 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086403560000000001, 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119813+0000", 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4950000000000001e-06 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119822+0000", 2024-09-16T12:59:22.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4210000000000001e-06 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119840+0000", 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7824000000000001e-05 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119848+0000", 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9750000000000007e-06 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128321+0000", 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084731100000000007 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128330+0000", 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619999999999998e-06 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128452+0000", 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012226899999999999 2024-09-16T12:59:22.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3620 4.16 4:6b60bac8:::rbd_header.11b5c378985b:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.213265+0000", 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.38318480199999999, 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029414599999999999, 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:22.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3620 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.213265+0000", 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.213265+0000", 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.213267+0000", 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8260000000000001e-06 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.213282+0000", 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4168000000000001e-05 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.213323+0000", 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1267999999999999e-05 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.213343+0000", 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0166999999999999e-05 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.213465+0000", 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012236199999999999 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.213630+0000", 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000164924 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.214077+0000", 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044669799999999998 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.215341+0000", 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012642199999999999 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.216054+0000", 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071274700000000001 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.216145+0000", 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1061000000000001e-05 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.216206+0000", 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1019e-05 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:22.722 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:22.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:22.724 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:22.719+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:22.825 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:22.815+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:22.830 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:22.830 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:22.830 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:22.830 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:22.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:22.847 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T12:59:22.925 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:22.915+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:22.936 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:22.936 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:22.936 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:22.936 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:22.936 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:22.946 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:22.957 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone3 2024-09-16T12:59:23.026 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:23.023+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:23.045 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-16T12:59:23.066 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:346 4.2 4:4550e1e5:::rbd_id.image_client.0-clone1:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.2072827559999997, 2024-09-16T12:59:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003768212, 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 346 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850713+0000", 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6270000000000001e-06 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850722+0000", 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2169999999999994e-06 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850739+0000", 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7005000000000001e-05 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850758+0000", 2024-09-16T12:59:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9502999999999999e-05 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850826+0000", 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8032000000000002e-05 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850959+0000", 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013249600000000001 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.851164+0000", 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020556 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.852079+0000", 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091475699999999998 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854331+0000", 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022523040000000001 2024-09-16T12:59:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854400+0000", 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8356999999999996e-05 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854480+0000", 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0353999999999997e-05 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:543 4.7 4:e655f1b3:::rbd_data.11b59d02cc38.000000000000004c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 367827~25389] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8226763430000004, 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004583953, 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 543 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235321+0000", 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.117e-06 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235354+0000", 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2962000000000001e-05 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235394+0000", 2024-09-16T12:59:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9954999999999999e-05 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235409+0000", 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4756999999999999e-05 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235553+0000", 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000144284 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235809+0000", 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025601299999999998 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.236081+0000", 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000272093 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237888+0000", 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806298 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239756+0000", 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001867884 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239842+0000", 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6581000000000003e-05 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239902+0000", 2024-09-16T12:59:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009000000000001e-05 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:556 4.7 4:e23e5852:::rbd_data.11b59d02cc38.0000000000000074:head [set-alloc-hint object_size 1048576 write_size 1048576,write 736209~50223] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7933337910000002, 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038485979999999999, 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 556 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264664+0000", 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264757+0000", 2024-09-16T12:59:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2904000000000004e-05 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264792+0000", 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4437000000000002e-05 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264804+0000", 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.218e-05 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264922+0000", 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118559 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265276+0000", 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035357999999999999 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265652+0000", 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037578600000000002 2024-09-16T12:59:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.267338+0000", 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016864899999999999 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268352+0000", 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00101337 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268425+0000", 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3398000000000004e-05 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268509+0000", 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4318000000000001e-05 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:574 4.7 4:e23e5852:::rbd_data.11b59d02cc38.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7457780229999997, 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051945439999999997, 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 574 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3300000000000003e-07 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312221+0000", 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9550000000000002e-06 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312229+0000", 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6159999999999996e-06 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312233+0000", 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8779999999999997e-06 2024-09-16T12:59:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312886+0000", 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065238099999999995 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313137+0000", 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025079300000000002 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313541+0000", 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040414999999999998 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314738+0000", 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011973839999999999 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317292+0000", 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002554312 2024-09-16T12:59:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317342+0000", 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9175999999999998e-05 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317411+0000", 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9466000000000002e-05 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:710 4.f 4:f1e0f236:::rbd_data.11b59d02cc38.000000000000008e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 694412~26484] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.688159+0000", 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3698353799999996, 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034345560000000001, 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 710 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688159+0000", 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688159+0000", 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688162+0000", 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.027e-06 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688195+0000", 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2824999999999998e-05 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688232+0000", 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6745000000000001e-05 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688251+0000", 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9670999999999999e-05 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688371+0000", 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119141 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688594+0000", 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022353100000000001 2024-09-16T12:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688799+0000", 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020529599999999999 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.690783+0000", 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019834330000000002 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691370+0000", 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058718900000000001 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691494+0000", 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012406299999999999 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691594+0000", 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9635000000000004e-05 2024-09-16T12:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:711 4.4 4:20cc0444:::rbd_data.11b59d02cc38.000000000000008f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 655360~35331] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3695865080000003, 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038704080000000001, 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 711 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688410+0000", 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688470+0000", 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0034000000000002e-05 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688491+0000", 2024-09-16T12:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605999999999999e-05 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688508+0000", 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7452000000000002e-05 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688639+0000", 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130541 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688984+0000", 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034521699999999998 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.689500+0000", 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000516023 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692011+0000", 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025114349999999998 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692150+0000", 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013872999999999999 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692223+0000", 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2917999999999995e-05 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692278+0000", 2024-09-16T12:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5402999999999998e-05 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:715 4.4 4:2570fb00:::rbd_data.11b59d02cc38.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3566961129999999, 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037942140000000002, 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 715 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701300+0000", 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9920000000000002e-06 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701311+0000", 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0361e-05 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701343+0000", 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2181000000000002e-05 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701361+0000", 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8366999999999999e-05 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701465+0000", 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103854 2024-09-16T12:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701709+0000", 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024390300000000001 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702064+0000", 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035441500000000001 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.704221+0000", 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021577609999999998 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.704960+0000", 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073849600000000005 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.705039+0000", 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9092999999999996e-05 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.705093+0000", 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3791e-05 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:719 4.4 4:2105cf60:::rbd_data.11b59d02cc38.0000000000000009:head [set-alloc-hint object_size 1048576 write_size 1048576,write 639060~16300] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.709251+0000", 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3487438520000001, 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032654839999999999, 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 719 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709251+0000", 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709251+0000", 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709253+0000", 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1969999999999999e-06 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709280+0000", 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7124999999999999e-05 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709315+0000", 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5228000000000002e-05 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709332+0000", 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7028e-05 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709449+0000", 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011715800000000001 2024-09-16T12:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709669+0000", 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021948000000000001 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709887+0000", 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021756799999999999 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712039+0000", 2024-09-16T12:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002152513 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712372+0000", 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033325999999999999 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712459+0000", 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6304000000000006e-05 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712516+0000", 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7623e-05 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:826 4.4 4:22c6e589:::rbd_data.11b59d02cc38.000000000000005a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 196608~7156] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1288442999999999, 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043112890000000003, 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 826 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929153+0000", 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.976e-06 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929175+0000", 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1897999999999999e-05 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929213+0000", 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7528999999999999e-05 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929230+0000", 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7093999999999999e-05 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929357+0000", 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012754300000000001 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929610+0000", 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025261899999999999 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929979+0000", 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036918900000000002 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.932855+0000", 2024-09-16T12:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028755880000000001 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933292+0000", 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043741500000000003 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933360+0000", 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7533999999999994e-05 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933462+0000", 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010190400000000001 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:827 4.4 4:2570fb00:::rbd_data.11b59d02cc38.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write 155625~40983] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1283431420000003, 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045348059999999997, 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 827 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929654+0000", 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-06 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929755+0000", 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100955 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929783+0000", 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7902000000000002e-05 2024-09-16T12:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929798+0000", 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5381000000000001e-05 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930965+0000", 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011667310000000001 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.931262+0000", 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029734500000000002 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.931625+0000", 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036316799999999998 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933825+0000", 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021993970000000001 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934061+0000", 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000236095 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934138+0000", 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7188999999999996e-05 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934186+0000", 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8303000000000002e-05 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:845 4.7 4:e7d4e431:::rbd_data.11b59d02cc38.0000000000000077:head [set-alloc-hint object_size 1048576 write_size 1048576,write 327680~11175] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.964326+0000", 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0936683660000002, 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003069188, 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 845 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964326+0000", 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964326+0000", 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964327+0000", 2024-09-16T12:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2279999999999999e-06 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964371+0000", 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3460000000000001e-05 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964397+0000", 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999999e-05 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964405+0000", 2024-09-16T12:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3429999999999996e-06 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964562+0000", 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015724700000000001 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964984+0000", 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042120299999999999 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.965431+0000", 2024-09-16T12:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044723799999999998 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967073+0000", 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016422240000000001 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967279+0000", 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020625400000000001 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967346+0000", 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6255999999999997e-05 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967395+0000", 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9774999999999997e-05 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:948 4.f 4:f1e0f236:::rbd_data.11b59d02cc38.000000000000008e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.8694338799999999, 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013393758, 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 948 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188563+0000", 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188576+0000", 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3071000000000001e-05 2024-09-16T12:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188611+0000", 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4623999999999999e-05 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188624+0000", 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2958e-05 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188732+0000", 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107911 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188873+0000", 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014080700000000001 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.192672+0000", 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037992270000000001 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201725+0000", 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090529549999999997 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201811+0000", 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6328000000000004e-05 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201894+0000", 2024-09-16T12:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2991000000000006e-05 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201954+0000", 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0336e-05 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:948 4.f e38/31 ondisk, result = 0)", 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.198624+0000", 2024-09-16T12:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.8593707669999997, 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033460849999999999, 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198624+0000", 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198624+0000", 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198625+0000", 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-06 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198630+0000", 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4650000000000002e-06 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198648+0000", 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7638999999999998e-05 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198656+0000", 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5950000000000001e-06 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201782+0000", 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031262239999999999 2024-09-16T12:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201795+0000", 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3121e-05 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201970+0000", 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017486800000000001 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:999 4.f 4:f13e1533:::rbd_data.11b59d02cc38.0000000000000035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7491236170000004, 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041771059999999999, 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 999 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308872+0000", 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.514e-06 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308880+0000", 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3300000000000001e-06 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308903+0000", 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3017999999999999e-05 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308915+0000", 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2636000000000001e-05 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309000+0000", 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4263000000000003e-05 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309082+0000", 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2443999999999999e-05 2024-09-16T12:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309190+0000", 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010767700000000001 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.310638+0000", 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001448509 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.312936+0000", 2024-09-16T12:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002297669 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.312990+0000", 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3494e-05 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.313048+0000", 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8551999999999998e-05 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1164 4.2 4:400c9634:::rbd_header.11b53140b009:head [watch watch cookie 140717154602240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2941396789999997, 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0048381040000000002, 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1164 2024-09-16T12:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763856+0000", 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.096e-06 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763861+0000", 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3199999999999999e-06 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763877+0000", 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.537e-05 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763885+0000", 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.687e-06 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763943+0000", 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7243000000000001e-05 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764001+0000", 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8214000000000002e-05 2024-09-16T12:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764123+0000", 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012186899999999999 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764972+0000", 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084946000000000002 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768511+0000", 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035386139999999998 2024-09-16T12:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768579+0000", 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8584999999999995e-05 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768693+0000", 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113646 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1268 4.2 4:40143cfe:::rbd_data.11b53140b009.00000000000000d2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.045972+0000", 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0120225270000001, 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030905780000000001, 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1268 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045972+0000", 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045972+0000", 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045975+0000", 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4429999999999999e-06 2024-09-16T12:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045991+0000", 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5475e-05 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046028+0000", 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7205e-05 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046046+0000", 2024-09-16T12:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7520000000000002e-05 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046145+0000", 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9779999999999997e-05 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046385+0000", 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023985099999999999 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046700+0000", 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000314452 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.047681+0000", 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098164099999999989 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049004+0000", 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013222539999999999 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049036+0000", 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2174000000000001e-05 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049063+0000", 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6783000000000001e-05 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1269 4.7 4:e145092b:::rbd_data.11b53140b009.00000000000000d3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0089660279999997, 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035041009999999999, 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1269 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2699999999999999e-07 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049033+0000", 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629999999999997e-06 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049041+0000", 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9920000000000007e-06 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049056+0000", 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4344e-05 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049131+0000", 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049343+0000", 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021202300000000001 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049705+0000", 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000362529 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.050871+0000", 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001165391 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052300+0000", 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014292459999999999 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052430+0000", 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013032399999999999 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052533+0000", 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102215 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1287 4.7 4:e145092b:::rbd_data.11b53140b009.00000000000000d3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.9931246410000001, 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042846999999999998, 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1287 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064871+0000", 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-07 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064875+0000", 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8029999999999996e-06 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064889+0000", 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3665e-05 2024-09-16T12:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064906+0000", 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7343e-05 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064971+0000", 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5147999999999994e-05 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.065076+0000", 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105004 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.065203+0000", 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126451 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.066904+0000", 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017010580000000001 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069097+0000", 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021933299999999998 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069130+0000", 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3161999999999999e-05 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069155+0000", 2024-09-16T12:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4161000000000001e-05 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1306 4.12 4:4f3fb25c:::rbd_data.11b53140b009.00000000000000c9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 147456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.9709947469999998, 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038175800000000001, 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1306 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087001+0000", 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7699999999999992e-07 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087006+0000", 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5010000000000004e-06 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087023+0000", 2024-09-16T12:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.662e-05 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087029+0000", 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4860000000000002e-06 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087105+0000", 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5265000000000005e-05 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087480+0000", 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037499899999999999 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087905+0000", 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042498600000000003 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.089239+0000", 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013342930000000001 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090707+0000", 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001468235 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090771+0000", 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3553000000000006e-05 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090817+0000", 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6665000000000001e-05 2024-09-16T12:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1307 4.7 4:e4cf6733:::rbd_data.11b53140b009.00000000000000ca:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 147456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.9709387199999999, 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003717405, 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1307 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087061+0000", 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8919999999999999e-06 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087069+0000", 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1470000000000001e-06 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087074+0000", 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2400000000000001e-06 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087159+0000", 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5765000000000003e-05 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087484+0000", 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032450900000000001 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087913+0000", 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042913400000000002 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.089457+0000", 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001544028 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090619+0000", 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011616090000000001 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090710+0000", 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1416999999999996e-05 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090773+0000", 2024-09-16T12:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3089999999999994e-05 2024-09-16T12:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:23.187 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:23.188 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:23.183+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:23.288 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:23.283+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:23.294 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:23.294 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:23.294 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:23.294 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:23.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:23.389 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:23.379+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:23.404 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:23.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:23.404 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:23.404 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:23.404 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:23.411 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:23.414 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:23.432 INFO:teuthology.orchestra.run.smithi042.stdout:423 write 0x38a12a0 thru 0x38a8c2e (0x798f bytes) 2024-09-16T12:59:23.436 INFO:teuthology.orchestra.run.smithi042.stdout:424 punch from 0x228d452 to 0x229c343, (0xeef1 bytes) 2024-09-16T12:59:23.437 INFO:teuthology.orchestra.run.smithi042.stdout:425 read 0x2cc1f9a thru 0x2cc8ab9 (0x6b20 bytes) 2024-09-16T12:59:23.437 INFO:teuthology.orchestra.run.smithi042.stdout:426 punch from 0x17b543e to 0x17b7979, (0x253b bytes) 2024-09-16T12:59:23.437 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:23.465 INFO:teuthology.orchestra.run.smithi042.stdout:429 write 0xd149362 thru 0xd158b39 (0xf7d8 bytes) 2024-09-16T12:59:23.471 INFO:teuthology.orchestra.run.smithi042.stdout:430 write 0xaa755ad thru 0xaa797e0 (0x4234 bytes) 2024-09-16T12:59:23.477 INFO:teuthology.orchestra.run.smithi042.stdout:431 write 0x5b1489d thru 0x5b24550 (0xfcb4 bytes) 2024-09-16T12:59:23.484 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:23.489 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:23.483+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:23.498 INFO:teuthology.orchestra.run.smithi042.stdout:433 trunc from 0xd158b3a to 0x5126272 2024-09-16T12:59:23.528 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7418496819999998, 2024-09-16T12:59:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003284908, 2024-09-16T12:59:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 10 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779657+0000", 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1239999999999999e-06 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779667+0000", 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0487e-05 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779683+0000", 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6146000000000001e-05 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779695+0000", 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1347e-05 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779769+0000", 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.449e-05 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779918+0000", 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000148462 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.780088+0000", 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017066600000000001 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.781019+0000", 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093024399999999999 2024-09-16T12:59:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782840+0000", 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018211379999999999 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782899+0000", 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8777000000000003e-05 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782940+0000", 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1026999999999999e-05 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:502 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 726006~2122] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3780817489999997, 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026083510000000001, 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 502 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143426+0000", 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0460000000000001e-06 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143465+0000", 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143507+0000", 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1678999999999999e-05 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143523+0000", 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6464e-05 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143631+0000", 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107357 2024-09-16T12:59:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143882+0000", 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025138100000000001 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.144130+0000", 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024822700000000001 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145527+0000", 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001396986 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145925+0000", 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000397964 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145985+0000", 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9896999999999999e-05 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.146031+0000", 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5938e-05 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:517 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.178491+0000", 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3430138439999997, 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002232773, 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 517 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178491+0000", 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178491+0000", 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178494+0000", 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-06 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178514+0000", 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9434999999999999e-05 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178564+0000", 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0507999999999998e-05 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178584+0000", 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9758e-05 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178747+0000", 2024-09-16T12:59:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016324199999999999 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.178980+0000", 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023328499999999999 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.179251+0000", 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027103400000000002 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.180615+0000", 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001363085 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.180638+0000", 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3465999999999999e-05 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.180684+0000", 2024-09-16T12:59:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5736999999999999e-05 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.180723+0000", 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9533000000000002e-05 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:622 4.d e37/31 4:b218b9b2:::rbd_data.11b59d02cc38.00000000000000e2:head v 37'3, mlcod=36'1)", 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.445836+0000", 2024-09-16T12:59:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.0756681160000001, 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020253850000000002, 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.445836+0000", 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.445836+0000", 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.445839+0000", 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0170000000000001e-06 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.445953+0000", 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000114162 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.445991+0000", 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7614999999999998e-05 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.446010+0000", 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9094e-05 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.446132+0000", 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121817 2024-09-16T12:59:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.446165+0000", 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3040999999999998e-05 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.447737+0000", 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001572033 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.447862+0000", 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012460600000000001 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:623 4.d e37/31 4:b218b9b2:::rbd_data.11b59d02cc38.00000000000000e2:head v 37'4, mlcod=36'2)", 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.450007+0000", 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.0714970409999998, 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020558099999999999, 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.450007+0000", 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.450007+0000", 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.450009+0000", 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5710000000000001e-06 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.450021+0000", 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2353e-05 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.450051+0000", 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9739000000000001e-05 2024-09-16T12:59:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.450069+0000", 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8484000000000001e-05 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.450176+0000", 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106995 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.450230+0000", 2024-09-16T12:59:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3260000000000002e-05 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.451973+0000", 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017429139999999999 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.452063+0000", 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0494000000000005e-05 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:681 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [set-alloc-hint object_size 1048576 write_size 1048576,write 119408~11664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.9129436120000003, 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035485299999999998, 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 681 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608564+0000", 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.146e-06 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608592+0000", 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8585000000000002e-05 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608635+0000", 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629000000000001e-05 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608649+0000", 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4270000000000001e-05 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608797+0000", 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014800499999999999 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609162+0000", 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036455100000000001 2024-09-16T12:59:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609411+0000", 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024895400000000002 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611008+0000", 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015968860000000001 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611948+0000", 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094050999999999998 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612013+0000", 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5185999999999996e-05 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612109+0000", 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5808000000000002e-05 2024-09-16T12:59:23.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:688 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.877613985, 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027962989999999999, 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 688 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643893+0000", 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8200000000000001e-06 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643906+0000", 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643953+0000", 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6399000000000001e-05 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643972+0000", 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9130000000000001e-05 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644163+0000", 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019125899999999999 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644426+0000", 2024-09-16T12:59:23.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026297899999999999 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644768+0000", 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000342574 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646240+0000", 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001471312 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646617+0000", 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037757399999999999 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646658+0000", 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1083e-05 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646687+0000", 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8248000000000001e-05 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:711 4.4 e37/31 4:20cc0444:::rbd_data.11b59d02cc38.000000000000008f:head v 37'3, mlcod=36'1)", 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.689773+0000", 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8317308959999998, 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021013500000000001, 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.689773+0000", 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.689773+0000", 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.689775+0000", 2024-09-16T12:59:23.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.685e-06 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.689846+0000", 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0687999999999998e-05 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.689876+0000", 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0008000000000001e-05 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.689889+0000", 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2945000000000001e-05 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.690015+0000", 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126668 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.690079+0000", 2024-09-16T12:59:23.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3973999999999994e-05 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691779+0000", 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016997430000000001 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691875+0000", 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5638999999999997e-05 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:715 4.4 e37/31 4:2570fb00:::rbd_data.11b59d02cc38.000000000000005b:head v 37'4, mlcod=36'2)", 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.702469+0000", 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8190348929999995, 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022212849999999999, 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702469+0000", 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702469+0000", 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702471+0000", 2024-09-16T12:59:23.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7540000000000001e-06 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702520+0000", 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9024999999999999e-05 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702552+0000", 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2236000000000001e-05 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702566+0000", 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3057000000000001e-05 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702673+0000", 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010791200000000001 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702725+0000", 2024-09-16T12:59:23.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1768999999999999e-05 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.704593+0000", 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00186773 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.704691+0000", 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7801999999999996e-05 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:719 4.4 e37/31 4:2105cf60:::rbd_data.11b59d02cc38.0000000000000009:head v 37'5, mlcod=37'3)", 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.710112+0000", 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8113924690000003, 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002006659, 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.710112+0000", 2024-09-16T12:59:23.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.710112+0000", 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.710114+0000", 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.79e-06 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.710164+0000", 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0593000000000002e-05 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.710195+0000", 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0518000000000002e-05 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.710209+0000", 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4239999999999999e-05 2024-09-16T12:59:23.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.710318+0000", 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108886 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.710352+0000", 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4307999999999997e-05 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712003+0000", 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001650983 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712119+0000", 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011534100000000001 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:810 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 443751~14451] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:23.582 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.6248600819999997, 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036011279999999999, 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 810 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896648+0000", 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6109999999999998e-06 2024-09-16T12:59:23.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896701+0000", 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3168999999999999e-05 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896743+0000", 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1445e-05 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896758+0000", 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5591e-05 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896878+0000", 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120256 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897013+0000", 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134327 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897221+0000", 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020804800000000001 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.899109+0000", 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018886899999999999 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900084+0000", 2024-09-16T12:59:23.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097495899999999996 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900178+0000", 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3264999999999996e-05 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900245+0000", 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7767000000000004e-05 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:822 4.d e38/31 4:b218b9b2:::rbd_data.11b59d02cc38.00000000000000e2:head v 38'7, mlcod=37'5)", 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.921249+0000", 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.6002556070000002, 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023282530000000002, 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921249+0000", 2024-09-16T12:59:23.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921249+0000", 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921249+0000", 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9699999999999996e-07 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921309+0000", 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9636000000000003e-05 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921334+0000", 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4691000000000001e-05 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921346+0000", 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1936999999999999e-05 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921466+0000", 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012074 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921511+0000", 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4659000000000001e-05 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.923480+0000", 2024-09-16T12:59:23.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001968914 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.923577+0000", 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7178999999999999e-05 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:826 4.4 e38/31 4:22c6e589:::rbd_data.11b59d02cc38.000000000000005a:head v 38'8, mlcod=37'6)", 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.591341527, 2024-09-16T12:59:23.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028983849999999999, 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930165+0000", 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.114e-06 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930191+0000", 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6057000000000001e-05 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930222+0000", 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0959e-05 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930235+0000", 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930349+0000", 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011372200000000001 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930402+0000", 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2766000000000003e-05 2024-09-16T12:59:23.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.932954+0000", 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025517980000000001 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933061+0000", 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010770800000000001 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:829 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.5796354140000002, 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024711590000000001, 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 829 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941872+0000", 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.225e-06 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941889+0000", 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6487999999999999e-05 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941926+0000", 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7727e-05 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941941+0000", 2024-09-16T12:59:23.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5085999999999999e-05 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942053+0000", 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011112600000000001 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942210+0000", 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000157653 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942387+0000", 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000176959 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.943529+0000", 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001141801 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944262+0000", 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073265199999999998 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944304+0000", 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2200999999999998e-05 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944340+0000", 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6241000000000001e-05 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.603 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:860 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [set-alloc-hint object_size 1048576 write_size 1048576,write 997016~9314] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.5232289540000004, 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037930170000000001, 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 860 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998280+0000", 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0969999999999999e-06 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998307+0000", 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7804999999999999e-05 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998349+0000", 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1838000000000002e-05 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998364+0000", 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5194000000000001e-05 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998511+0000", 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014693900000000001 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998912+0000", 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000401066 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.999376+0000", 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000463938 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001244+0000", 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018672389999999999 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001920+0000", 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067617100000000004 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002008+0000", 2024-09-16T12:59:23.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8342000000000002e-05 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002068+0000", 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0387999999999998e-05 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:995 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:23.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.2143478610000003, 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040928859999999996, 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 995 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307159+0000", 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9739999999999998e-06 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307174+0000", 2024-09-16T12:59:23.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4134000000000001e-05 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307211+0000", 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7385000000000002e-05 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307230+0000", 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8961e-05 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307461+0000", 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230721 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307710+0000", 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024951200000000002 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308016+0000", 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030602199999999998 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309601+0000", 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015845200000000001 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311094+0000", 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00149321 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311180+0000", 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6142000000000003e-05 2024-09-16T12:59:23.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311249+0000", 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9305000000000001e-05 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3466 4.4 4:2737304d:::rbd_data.11b5c378985b.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1448886~47421] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:21.917059+0000", 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.6044455960000001, 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024009790000000001, 2024-09-16T12:59:23.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3466 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917059+0000", 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917059+0000", 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917061+0000", 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.829e-06 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917120+0000", 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9011000000000001e-05 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917146+0000", 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6259e-05 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917154+0000", 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4980000000000006e-06 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917214+0000", 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9806999999999998e-05 2024-09-16T12:59:23.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917373+0000", 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015901199999999999 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917544+0000", 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000171285 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.918579+0000", 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001034863 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.919356+0000", 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077711399999999997 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.919410+0000", 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3696000000000002e-05 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.919460+0000", 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9604999999999997e-05 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3511 4.4 4:2737304d:::rbd_data.11b5c378985b.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.007947+0000", 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.51355741, 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002085482, 2024-09-16T12:59:23.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3511 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007947+0000", 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007947+0000", 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007950+0000", 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0699999999999998e-06 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007961+0000", 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1008e-05 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007990+0000", 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9201999999999999e-05 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.008003+0000", 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2604e-05 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.008064+0000", 2024-09-16T12:59:23.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1511000000000001e-05 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.008143+0000", 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8695999999999993e-05 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.008248+0000", 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104654 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.009591+0000", 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013428170000000001 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.009836+0000", 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024574900000000002 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.009931+0000", 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4504000000000002e-05 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.010032+0000", 2024-09-16T12:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101667 2024-09-16T12:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3720 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.62637921500000004, 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024764850000000001, 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3720 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:23.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895128+0000", 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895141+0000", 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2824e-05 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895170+0000", 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8992e-05 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895181+0000", 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1106e-05 2024-09-16T12:59:23.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895274+0000", 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3073999999999997e-05 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895417+0000", 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014300599999999999 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895810+0000", 2024-09-16T12:59:23.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039280400000000002 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897118+0000", 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00130792 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897512+0000", 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039389699999999999 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897562+0000", 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0229000000000003e-05 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897602+0000", 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9892999999999999e-05 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3724 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.610709049, 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002592624, 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.634 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:23.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910798+0000", 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7350000000000001e-06 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910813+0000", 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4469e-05 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910847+0000", 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3986999999999998e-05 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910869+0000", 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2764000000000001e-05 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910939+0000", 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0086000000000006e-05 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.911119+0000", 2024-09-16T12:59:23.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179598 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.911351+0000", 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023168200000000001 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.912688+0000", 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013376729999999999 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913294+0000", 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060521299999999995 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913343+0000", 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9537999999999999e-05 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913388+0000", 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4879000000000002e-05 2024-09-16T12:59:23.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:23.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:23.640 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:23.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:23.641 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:23.635+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:23.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:23.643 INFO:teuthology.orchestra.run.smithi042.stdout:434 write 0x922dc5d thru 0x9236915 (0x8cb9 bytes) 2024-09-16T12:59:23.643 INFO:teuthology.orchestra.run.smithi042.stdout:436 clone 5 order 25 su 2097152 sc 5 2024-09-16T12:59:23.743 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:23.739+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:23.793 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:23.793 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:23.803 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:23.823 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:23.843 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:23.835+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:23.846 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:23.867 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:23.886 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:23.913 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:23.913 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:23.923 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:23.944 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:23.939+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:23.962 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone4 intact 2024-09-16T12:59:24.045 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:24.035+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:24.074 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:24.074 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:24.084 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:24.113 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:24.129 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-16T12:59:24.139 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:24.145 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:24.135+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:24.162 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:24.178 INFO:teuthology.orchestra.run.smithi042.stdout:437 read 0x28bfb3c thru 0x28cd262 (0xd727 bytes) 2024-09-16T12:59:24.185 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:24.187 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:24.187 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:24.188 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:24.188 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:24.188 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:24.188 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:24.188 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:24.188 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:24.196 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:24.197 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:24.203 INFO:teuthology.orchestra.run.smithi042.stdout:438 write 0xd1774ee thru 0xd17c8e1 (0x53f4 bytes) 2024-09-16T12:59:24.206 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:24.209 INFO:teuthology.orchestra.run.smithi042.stdout:440 writesame 0x7ae4377 thru 0x7aed762 data_size 0x2(0x93ec bytes) 2024-09-16T12:59:24.231 INFO:teuthology.orchestra.run.smithi042.stdout:441 write 0x8583c09 thru 0x858f61c (0xba14 bytes) 2024-09-16T12:59:24.234 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:24.234 INFO:teuthology.orchestra.run.smithi042.stdout:445 punch from 0x6ebb91a to 0x6ec9dce, (0xe4b4 bytes) 2024-09-16T12:59:24.234 INFO:teuthology.orchestra.run.smithi042.stdout:446 writesame 0xc8f86a9 thru 0xc8f9c19 data_size 0x1(0x1571 bytes) 2024-09-16T12:59:24.242 INFO:teuthology.orchestra.run.smithi042.stdout:449 punch from 0x1236b52 to 0x124243e, (0xb8ec bytes) 2024-09-16T12:59:24.246 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:24.239+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:24.257 INFO:teuthology.orchestra.run.smithi042.stdout:450 write 0xea4f1c6 thru 0xea510b3 (0x1eee bytes) 2024-09-16T12:59:24.266 INFO:teuthology.orchestra.run.smithi042.stdout:451 trunc from 0xea510b4 to 0xdbb928e 2024-09-16T12:59:24.306 INFO:teuthology.orchestra.run.smithi042.stdout:453 trunc from 0xdbb928e to 0x190cdd6 2024-09-16T12:59:24.311 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:24.311 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:24.322 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:24.366 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:24.363+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:24.397 INFO:teuthology.orchestra.run.smithi042.stdout:454 trunc from 0x190cdd6 to 0xcd4a52c 2024-09-16T12:59:24.408 INFO:teuthology.orchestra.run.smithi042.stdout:455 read 0x7ac2913 thru 0x7ac7e3f (0x552d bytes) 2024-09-16T12:59:24.409 INFO:teuthology.orchestra.run.smithi042.stdout:456 read 0x25ddb87 thru 0x25e33ca (0x5844 bytes) 2024-09-16T12:59:24.418 INFO:teuthology.orchestra.run.smithi042.stdout:457 trunc from 0xcd4a52c to 0x8f09996 2024-09-16T12:59:24.418 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:24.418 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:24.429 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:24.447 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:24.439+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:24.453 INFO:teuthology.orchestra.run.smithi042.stdout:459 write 0xd00a0a2 thru 0xd013e76 (0x9dd5 bytes) 2024-09-16T12:59:24.472 INFO:teuthology.orchestra.run.smithi042.stdout:460 trunc from 0xd013e77 to 0x810f219 2024-09-16T12:59:24.511 INFO:teuthology.orchestra.run.smithi042.stdout:461 trunc from 0x810f219 to 0xd97eacb 2024-09-16T12:59:24.522 INFO:teuthology.orchestra.run.smithi042.stdout:462 writesame 0x92530a7 thru 0x92552a8 data_size 0x3(0x2202 bytes) 2024-09-16T12:59:24.527 INFO:teuthology.orchestra.run.smithi042.stdout:463 punch from 0xa1c8fd9 to 0xa1cd0e0, (0x4107 bytes) 2024-09-16T12:59:24.527 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:24.527 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xec897fd 2024-09-16T12:59:24.528 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:24.528 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:24.529 INFO:teuthology.orchestra.run.smithi042.stdout:466 trunc from 0xd97eacb to 0xec897fd 2024-09-16T12:59:24.538 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:24.542 INFO:teuthology.orchestra.run.smithi042.stdout:467 punch from 0x8c71ece to 0x8c754e0, (0x3612 bytes) 2024-09-16T12:59:24.542 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:24.542 INFO:teuthology.orchestra.run.smithi042.stdout:469 read 0xe8877b5 thru 0xe88d941 (0x618d bytes) 2024-09-16T12:59:24.542 INFO:teuthology.orchestra.run.smithi042.stdout:470 write 0x524dcbb thru 0x525da54 (0xfd9a bytes) 2024-09-16T12:59:24.546 INFO:teuthology.orchestra.run.smithi042.stdout:473 read 0x4ae988a thru 0x4af2299 (0x8a10 bytes) 2024-09-16T12:59:24.569 INFO:teuthology.orchestra.run.smithi042.stdout:476 trunc from 0xec897fd to 0x46e44e3 2024-09-16T12:59:24.583 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:24.579+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:24.583 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:24.599 INFO:teuthology.orchestra.run.smithi042.stdout:479 write 0x657506d thru 0x6575861 (0x7f5 bytes) 2024-09-16T12:59:24.602 INFO:teuthology.orchestra.run.smithi042.stdout:480 read 0x1b43bc2 thru 0x1b44704 (0xb43 bytes) 2024-09-16T12:59:24.603 INFO:teuthology.orchestra.run.smithi042.stdout:481 read 0x23a1f63 thru 0x23adf6e (0xc00c bytes) 2024-09-16T12:59:24.604 INFO:teuthology.orchestra.run.smithi042.stdout:482 write 0xdb1e51 thru 0xdb7a85 (0x5c35 bytes) 2024-09-16T12:59:24.608 INFO:teuthology.orchestra.run.smithi042.stdout:483 writesame 0x4a69df thru 0x4b1380 data_size 0x1(0xa9a2 bytes) 2024-09-16T12:59:24.637 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:24.638 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:24.642 INFO:teuthology.orchestra.run.smithi042.stdout:484 punch from 0xf8cacd to 0xf95ac5, (0x8ff8 bytes) 2024-09-16T12:59:24.642 INFO:teuthology.orchestra.run.smithi042.stdout:486 read 0x636a8ad thru 0x636ecdf (0x4433 bytes) 2024-09-16T12:59:24.643 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:24.648 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:24.649 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:24.639+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:24.668 INFO:teuthology.orchestra.run.smithi042.stdout:489 write 0xc3fe179 thru 0xc3ffca4 (0x1b2c bytes) 2024-09-16T12:59:24.674 INFO:teuthology.orchestra.run.smithi042.stdout:490 punch from 0x28b926e to 0x28c0127, (0x6eb9 bytes) 2024-09-16T12:59:24.674 INFO:teuthology.orchestra.run.smithi042.stdout:491 write 0x81bf48f thru 0x81c147e (0x1ff0 bytes) 2024-09-16T12:59:24.677 INFO:teuthology.orchestra.run.smithi042.stdout:492 punch from 0x6ce04de to 0x6ce35fd, (0x311f bytes) 2024-09-16T12:59:24.677 INFO:teuthology.orchestra.run.smithi042.stdout:493 read 0x53cb93 thru 0x54833f (0xb7ad bytes) 2024-09-16T12:59:24.678 INFO:teuthology.orchestra.run.smithi042.stdout:495 writesame 0x86369be thru 0x8639f68 data_size 0x1(0x35ab bytes) 2024-09-16T12:59:24.690 INFO:teuthology.orchestra.run.smithi042.stdout:497 write 0x8ea8aa0 thru 0x8eac112 (0x3673 bytes) 2024-09-16T12:59:24.693 INFO:teuthology.orchestra.run.smithi042.stdout:499 read 0x1210a60 thru 0x1213fea (0x358b bytes) 2024-09-16T12:59:24.693 INFO:teuthology.orchestra.run.smithi042.stdout:500 write 0x9f5bd57 thru 0x9f611e1 (0x548b bytes) 2024-09-16T12:59:24.697 INFO:teuthology.orchestra.run.smithi042.stdout:501 punch from 0x68e6e13 to 0x68ea8da, (0x3ac7 bytes) 2024-09-16T12:59:24.708 INFO:teuthology.orchestra.run.smithi042.stdout:503 write 0xc66060f thru 0xc66a54e (0x9f40 bytes) 2024-09-16T12:59:24.715 INFO:teuthology.orchestra.run.smithi042.stdout:504 write 0x622d2a5 thru 0x6236f20 (0x9c7c bytes) 2024-09-16T12:59:24.718 INFO:teuthology.orchestra.run.smithi042.stdout:505 punch from 0xc588e7b to 0xc597d3d, (0xeec2 bytes) 2024-09-16T12:59:24.718 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:24.719 INFO:teuthology.orchestra.run.smithi042.stdout:507 write 0x977a45c thru 0x977c5ee (0x2193 bytes) 2024-09-16T12:59:24.722 INFO:teuthology.orchestra.run.smithi042.stdout:509 write 0xb3591d5 thru 0xb36272b (0x9557 bytes) 2024-09-16T12:59:24.743 INFO:teuthology.orchestra.run.smithi042.stdout:510 write 0xd6200e1 thru 0xd6293ea (0x930a bytes) 2024-09-16T12:59:24.750 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:24.747+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:24.751 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:24.751 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:24.760 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:24.770 INFO:teuthology.orchestra.run.smithi042.stdout:511 trunc from 0xd6293eb to 0x31eb645 2024-09-16T12:59:24.811 INFO:teuthology.orchestra.run.smithi042.stdout:513 write 0x5031bd3 thru 0x503ae7c (0x92aa bytes) 2024-09-16T12:59:24.816 INFO:teuthology.orchestra.run.smithi042.stdout:514 trunc from 0x503ae7d to 0x47c104d 2024-09-16T12:59:24.832 INFO:teuthology.orchestra.run.smithi042.stdout:515 read 0x44859c1 thru 0x448db5d (0x819d bytes) 2024-09-16T12:59:24.842 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T12:59:24.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T12:59:24.856 INFO:teuthology.orchestra.run.smithi042.stdout:517 writesame 0x97e47cb thru 0x97e8fef data_size 0x17(0x4825 bytes) 2024-09-16T12:59:24.856 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:24.847+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:24.858 INFO:teuthology.orchestra.run.smithi042.stdout:518 clone 6 order 23 su 4194304 sc 5 2024-09-16T12:59:24.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T12:59:24.867 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:24.867 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:24.876 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:24.952 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:24.943+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:24.971 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:24.972 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:24.981 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:25.053 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:25.047+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:25.079 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:25.079 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:25.089 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:25.154 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:25.143+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:25.186 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:25.186 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:25.195 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:25.255 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:25.251+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:25.296 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:25.296 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:25.305 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:25.356 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:25.351+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:25.445 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:25.445 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:25.455 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:25.456 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:25.451+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:25.557 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:25.551+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:25.564 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:25.565 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:25.574 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:25.658 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:25.655+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:25.672 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:25.672 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:25.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:25.759 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:25.751+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:25.787 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:25.787 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:25.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:25.859 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:25.851+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:25.861 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T12:59:25.862 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T12:59:25.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T12:59:25.944 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:25.944 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:25.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:25.960 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:25.955+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:25.981 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone5 from 0x97e8ff0 (overlap 0x190cdd6) to 0x0 2024-09-16T12:59:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:26.061 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:26.051+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:26.068 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:26.111 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-16T12:59:26.162 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:26.155+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:26.181 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:26.181 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:26.192 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:26.198 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:26.263 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:26.259+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:26.288 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:26.288 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:26.298 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:26.342 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:26.364 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:26.355+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:26.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:26.391 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:26.391 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:26.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:26.401 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:26.416 INFO:teuthology.orchestra.run.smithi042.stdout:519 writesame 0x626094c thru 0x6265631 data_size 0x33(0x4ce6 bytes) 2024-09-16T12:59:26.426 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:26.433 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:26.433 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:26.433 INFO:teuthology.orchestra.run.smithi042.stdout:523 clone 7 order 25 su 16777216 sc 2 2024-09-16T12:59:26.448 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:26.465 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:26.455+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:26.475 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:26.497 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:26.506 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.719209055, 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003284908, 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 10 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779657+0000", 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1239999999999999e-06 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779667+0000", 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0487e-05 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779683+0000", 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6146000000000001e-05 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779695+0000", 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1347e-05 2024-09-16T12:59:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779769+0000", 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.449e-05 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779918+0000", 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000148462 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.780088+0000", 2024-09-16T12:59:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017066600000000001 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.781019+0000", 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093024399999999999 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782840+0000", 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018211379999999999 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782899+0000", 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8777000000000003e-05 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782940+0000", 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1026999999999999e-05 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:502 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 726006~2122] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3554411220000002, 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026083510000000001, 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 502 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143426+0000", 2024-09-16T12:59:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0460000000000001e-06 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143465+0000", 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143507+0000", 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1678999999999999e-05 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143523+0000", 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6464e-05 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143631+0000", 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107357 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143882+0000", 2024-09-16T12:59:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025138100000000001 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.144130+0000", 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024822700000000001 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145527+0000", 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001396986 2024-09-16T12:59:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145925+0000", 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000397964 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145985+0000", 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9896999999999999e-05 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.146031+0000", 2024-09-16T12:59:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5938e-05 2024-09-16T12:59:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:681 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [set-alloc-hint object_size 1048576 write_size 1048576,write 119408~11664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.8903029849999999, 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035485299999999998, 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 681 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608564+0000", 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.146e-06 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608592+0000", 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8585000000000002e-05 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608635+0000", 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629000000000001e-05 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608649+0000", 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4270000000000001e-05 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608797+0000", 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014800499999999999 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609162+0000", 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036455100000000001 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609411+0000", 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024895400000000002 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611008+0000", 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015968860000000001 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611948+0000", 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094050999999999998 2024-09-16T12:59:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612013+0000", 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5185999999999996e-05 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612109+0000", 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5808000000000002e-05 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:688 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.8549733580000005, 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027962989999999999, 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 688 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643893+0000", 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8200000000000001e-06 2024-09-16T12:59:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643906+0000", 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643953+0000", 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6399000000000001e-05 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643972+0000", 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9130000000000001e-05 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644163+0000", 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019125899999999999 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644426+0000", 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026297899999999999 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644768+0000", 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000342574 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646240+0000", 2024-09-16T12:59:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001471312 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646617+0000", 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037757399999999999 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646658+0000", 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1083e-05 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646687+0000", 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8248000000000001e-05 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:810 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 443751~14451] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6022194550000002, 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036011279999999999, 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 810 2024-09-16T12:59:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896648+0000", 2024-09-16T12:59:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6109999999999998e-06 2024-09-16T12:59:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896701+0000", 2024-09-16T12:59:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3168999999999999e-05 2024-09-16T12:59:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896743+0000", 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1445e-05 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896758+0000", 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5591e-05 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896878+0000", 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120256 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897013+0000", 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134327 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897221+0000", 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020804800000000001 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.899109+0000", 2024-09-16T12:59:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018886899999999999 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900084+0000", 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097495899999999996 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900178+0000", 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3264999999999996e-05 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900245+0000", 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7767000000000004e-05 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:822 4.d e38/31 4:b218b9b2:::rbd_data.11b59d02cc38.00000000000000e2:head v 38'7, mlcod=37'5)", 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.921249+0000", 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.5776149799999999, 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023282530000000002, 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921249+0000", 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921249+0000", 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921249+0000", 2024-09-16T12:59:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9699999999999996e-07 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921309+0000", 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9636000000000003e-05 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921334+0000", 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4691000000000001e-05 2024-09-16T12:59:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921346+0000", 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1936999999999999e-05 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921466+0000", 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012074 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.921511+0000", 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4659000000000001e-05 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.923480+0000", 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001968914 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.923577+0000", 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7178999999999999e-05 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:826 4.4 e38/31 4:22c6e589:::rbd_data.11b59d02cc38.000000000000005a:head v 38'8, mlcod=37'6)", 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.5687008999999996, 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028983849999999999, 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930165+0000", 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.114e-06 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930191+0000", 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6057000000000001e-05 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930222+0000", 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0959e-05 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930235+0000", 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930349+0000", 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011372200000000001 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930402+0000", 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2766000000000003e-05 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.932954+0000", 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025517980000000001 2024-09-16T12:59:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933061+0000", 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010770800000000001 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:829 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.5569947870000007, 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024711590000000001, 2024-09-16T12:59:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 829 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941872+0000", 2024-09-16T12:59:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.225e-06 2024-09-16T12:59:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941889+0000", 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6487999999999999e-05 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941926+0000", 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7727e-05 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941941+0000", 2024-09-16T12:59:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5085999999999999e-05 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942053+0000", 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011112600000000001 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942210+0000", 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000157653 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942387+0000", 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000176959 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.943529+0000", 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001141801 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944262+0000", 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073265199999999998 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944304+0000", 2024-09-16T12:59:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2200999999999998e-05 2024-09-16T12:59:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944340+0000", 2024-09-16T12:59:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6241000000000001e-05 2024-09-16T12:59:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:860 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [set-alloc-hint object_size 1048576 write_size 1048576,write 997016~9314] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.5005883269999991, 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037930170000000001, 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 860 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998280+0000", 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0969999999999999e-06 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998307+0000", 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7804999999999999e-05 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998349+0000", 2024-09-16T12:59:26.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1838000000000002e-05 2024-09-16T12:59:26.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998364+0000", 2024-09-16T12:59:26.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5194000000000001e-05 2024-09-16T12:59:26.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998511+0000", 2024-09-16T12:59:26.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014693900000000001 2024-09-16T12:59:26.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998912+0000", 2024-09-16T12:59:26.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000401066 2024-09-16T12:59:26.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.999376+0000", 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000463938 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001244+0000", 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018672389999999999 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001920+0000", 2024-09-16T12:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067617100000000004 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002008+0000", 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8342000000000002e-05 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002068+0000", 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0387999999999998e-05 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:995 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:26.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:26.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.1917072340000008, 2024-09-16T12:59:26.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040928859999999996, 2024-09-16T12:59:26.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 995 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307159+0000", 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9739999999999998e-06 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307174+0000", 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4134000000000001e-05 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307211+0000", 2024-09-16T12:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7385000000000002e-05 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307230+0000", 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8961e-05 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307461+0000", 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230721 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307710+0000", 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024951200000000002 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308016+0000", 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030602199999999998 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309601+0000", 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015845200000000001 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311094+0000", 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00149321 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311180+0000", 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6142000000000003e-05 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311249+0000", 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9305000000000001e-05 2024-09-16T12:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3466 4.4 4:2737304d:::rbd_data.11b5c378985b.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1448886~47421] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:21.917059+0000", 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5818049690000002, 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024009790000000001, 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3466 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917059+0000", 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917059+0000", 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917061+0000", 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.829e-06 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917120+0000", 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9011000000000001e-05 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917146+0000", 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6259e-05 2024-09-16T12:59:26.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917154+0000", 2024-09-16T12:59:26.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4980000000000006e-06 2024-09-16T12:59:26.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917214+0000", 2024-09-16T12:59:26.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9806999999999998e-05 2024-09-16T12:59:26.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917373+0000", 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015901199999999999 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.917544+0000", 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000171285 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.918579+0000", 2024-09-16T12:59:26.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001034863 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.919356+0000", 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077711399999999997 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.919410+0000", 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3696000000000002e-05 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.919460+0000", 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9604999999999997e-05 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3720 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.6037385880000001, 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024764850000000001, 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3720 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:26.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895128+0000", 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895141+0000", 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2824e-05 2024-09-16T12:59:26.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895170+0000", 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8992e-05 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895181+0000", 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1106e-05 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895274+0000", 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3073999999999997e-05 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895417+0000", 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014300599999999999 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895810+0000", 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039280400000000002 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897118+0000", 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00130792 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897512+0000", 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039389699999999999 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897562+0000", 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0229000000000003e-05 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897602+0000", 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9892999999999999e-05 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3724 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5880684220000001, 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002592624, 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910798+0000", 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7350000000000001e-06 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910813+0000", 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4469e-05 2024-09-16T12:59:26.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910847+0000", 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3986999999999998e-05 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910869+0000", 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2764000000000001e-05 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910939+0000", 2024-09-16T12:59:26.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0086000000000006e-05 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.911119+0000", 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179598 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.911351+0000", 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023168200000000001 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.912688+0000", 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013376729999999999 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913294+0000", 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060521299999999995 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913343+0000", 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9537999999999999e-05 2024-09-16T12:59:26.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913388+0000", 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4879000000000002e-05 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4040 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.036538701, 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026268979999999999, 2024-09-16T12:59:26.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4040 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462327+0000", 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0229999999999999e-06 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462340+0000", 2024-09-16T12:59:26.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3189e-05 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462373+0000", 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2555000000000003e-05 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462386+0000", 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2935000000000001e-05 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462458+0000", 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1847999999999994e-05 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462582+0000", 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012399199999999999 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.463055+0000", 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047307700000000002 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464541+0000", 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014867649999999999 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464841+0000", 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029975500000000002 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464912+0000", 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0344000000000003e-05 2024-09-16T12:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464952+0000", 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0414999999999998e-05 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4042 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.0301773980000002, 2024-09-16T12:59:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002527208, 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4042 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468688+0000", 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9549999999999999e-06 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468700+0000", 2024-09-16T12:59:26.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1443e-05 2024-09-16T12:59:26.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468726+0000", 2024-09-16T12:59:26.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6591999999999999e-05 2024-09-16T12:59:26.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468737+0000", 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0951999999999999e-05 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468841+0000", 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010335 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468995+0000", 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015392800000000001 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.469556+0000", 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00056112700000000002 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471063+0000", 2024-09-16T12:59:26.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001507226 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471130+0000", 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6693999999999995e-05 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471176+0000", 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6223000000000001e-05 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471214+0000", 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7718e-05 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4044 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.0244096850000002, 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002965217, 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4044 2024-09-16T12:59:26.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:26.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474456+0000", 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.818e-06 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474465+0000", 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5459999999999997e-06 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474490+0000", 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5058e-05 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474502+0000", 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1110999999999999e-05 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474610+0000", 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108606 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474815+0000", 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020453800000000001 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.475681+0000", 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086620700000000002 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477079+0000", 2024-09-16T12:59:26.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001398388 2024-09-16T12:59:26.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477340+0000", 2024-09-16T12:59:26.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026078800000000002 2024-09-16T12:59:26.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477385+0000", 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5014e-05 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477419+0000", 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4143000000000002e-05 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4046 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:26.606 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.495211+0000", 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.0036524440000001, 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024109330000000001, 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4046 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495211+0000", 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495211+0000", 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495215+0000", 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3009999999999998e-06 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495233+0000", 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8804000000000002e-05 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495275+0000", 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1634999999999997e-05 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495290+0000", 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.526e-05 2024-09-16T12:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495354+0000", 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3417000000000006e-05 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495455+0000", 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010140399999999999 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495831+0000", 2024-09-16T12:59:26.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037579699999999998 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.497137+0000", 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013057310000000001 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.497549+0000", 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000412438 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.497590+0000", 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1297000000000001e-05 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.497622+0000", 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1848999999999999e-05 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4052 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.506650+0000", 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.992213703, 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00230664, 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4052 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.506650+0000", 2024-09-16T12:59:26.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.506650+0000", 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.506652+0000", 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.334e-06 2024-09-16T12:59:26.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.506664+0000", 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1301000000000001e-05 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.506690+0000", 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6463999999999999e-05 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.506717+0000", 2024-09-16T12:59:26.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6375000000000001e-05 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.506810+0000", 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3239999999999995e-05 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.506981+0000", 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017097999999999999 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:26.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.507483+0000", 2024-09-16T12:59:26.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00050213799999999996 2024-09-16T12:59:26.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.508533+0000", 2024-09-16T12:59:26.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001049946 2024-09-16T12:59:26.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.508846+0000", 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031365100000000002 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.508912+0000", 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5716999999999994e-05 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.508957+0000", 2024-09-16T12:59:26.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4493999999999998e-05 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4060 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.523984+0000", 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.9748799090000002, 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023103099999999999, 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4060 2024-09-16T12:59:26.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.620 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.523984+0000", 2024-09-16T12:59:26.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.523984+0000", 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.523987+0000", 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1480000000000002e-06 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.524001+0000", 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4241e-05 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.524031+0000", 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9691000000000001e-05 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.524042+0000", 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0772e-05 2024-09-16T12:59:26.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.524120+0000", 2024-09-16T12:59:26.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-16T12:59:26.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.524247+0000", 2024-09-16T12:59:26.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127174 2024-09-16T12:59:26.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:26.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.524542+0000", 2024-09-16T12:59:26.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029473500000000002 2024-09-16T12:59:26.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.525685+0000", 2024-09-16T12:59:26.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001142905 2024-09-16T12:59:26.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.526142+0000", 2024-09-16T12:59:26.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045670699999999999 2024-09-16T12:59:26.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.526232+0000", 2024-09-16T12:59:26.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0805999999999998e-05 2024-09-16T12:59:26.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.526294+0000", 2024-09-16T12:59:26.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1793999999999998e-05 2024-09-16T12:59:26.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.627 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.628 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4069 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:26.628 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:26.628 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.9630347810000002, 2024-09-16T12:59:26.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025697279999999999, 2024-09-16T12:59:26.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:26.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:26.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:26.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:26.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:26.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4069 2024-09-16T12:59:26.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.630 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:26.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:26.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:26.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:26.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:26.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:26.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:26.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535831+0000", 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8109999999999999e-06 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535841+0000", 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0267e-05 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535862+0000", 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0693e-05 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535869+0000", 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7309999999999993e-06 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535958+0000", 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8721000000000007e-05 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.536094+0000", 2024-09-16T12:59:26.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013532999999999999 2024-09-16T12:59:26.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:26.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.536561+0000", 2024-09-16T12:59:26.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000467057 2024-09-16T12:59:26.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:26.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538128+0000", 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001567337 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538285+0000", 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015667900000000001 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538355+0000", 2024-09-16T12:59:26.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9991000000000001e-05 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538399+0000", 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4110999999999998e-05 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:26.636 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:26.638 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:26.638 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:26.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:26.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:26.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:26.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:26.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:26.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:26.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:26.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:26.640 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:26.635+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:26.738 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:26.735+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:26.785 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:26.785 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:26.785 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:26.785 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:26.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:26.839 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:26.835+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:26.875 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T12:59:26.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T12:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:26.909 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:26.940 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:26.931+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:27.012 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:27.012 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:441 4.0 4:07ec4131:::rbd_data.11b59d02cc38.00000000000000d3:head [set-alloc-hint object_size 1048576 write_size 1048576,write 0~15429] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.008323+0000", 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9960100470000004, 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039226770000000003, 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 441 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008323+0000", 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008323+0000", 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008326+0000", 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.841e-06 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008380+0000", 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4138999999999998e-05 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008431+0000", 2024-09-16T12:59:27.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1532000000000001e-05 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008483+0000", 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2077000000000003e-05 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008636+0000", 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015205000000000001 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.008927+0000", 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029145199999999999 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.009454+0000", 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000526999 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.011277+0000", 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018226480000000001 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.012144+0000", 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086761499999999997 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.012196+0000", 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1437999999999998e-05 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.012246+0000", 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9886000000000003e-05 2024-09-16T12:59:27.018 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:544 4.17 4:efdedeac:::rbd_data.11b59d02cc38.000000000000004d:head [set-alloc-hint object_size 1048576 write_size 1048576,write 327680~28801] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.237694+0000", 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.766639133, 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033377670000000002, 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.019 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 544 2024-09-16T12:59:27.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.020 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237694+0000", 2024-09-16T12:59:27.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237694+0000", 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237698+0000", 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7629999999999998e-06 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237757+0000", 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9382999999999997e-05 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237796+0000", 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9038999999999997e-05 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237810+0000", 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4347e-05 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237941+0000", 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013115699999999999 2024-09-16T12:59:27.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.238194+0000", 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000252482 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.238462+0000", 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026800100000000002 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.240549+0000", 2024-09-16T12:59:27.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020871309999999999 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.240820+0000", 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027090000000000003 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.240914+0000", 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4087999999999998e-05 2024-09-16T12:59:27.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.241032+0000", 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117476 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:557 4.17 4:eab06920:::rbd_data.11b59d02cc38.0000000000000075:head [set-alloc-hint object_size 1048576 write_size 1048576,write 720896~5966] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.265124+0000", 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7392088950000009, 2024-09-16T12:59:27.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003336604, 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 557 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265124+0000", 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265124+0000", 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265127+0000", 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.469e-06 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265144+0000", 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7050000000000001e-05 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265175+0000", 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1779000000000002e-05 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265187+0000", 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1516e-05 2024-09-16T12:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265303+0000", 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116334 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265615+0000", 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031190800000000001 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265973+0000", 2024-09-16T12:59:27.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035811600000000001 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.267565+0000", 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001591561 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268334+0000", 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076968299999999996 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268407+0000", 2024-09-16T12:59:27.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2152999999999998e-05 2024-09-16T12:59:27.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268461+0000", 2024-09-16T12:59:27.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4035e-05 2024-09-16T12:59:27.031 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.031 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.031 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:575 4.17 4:eab06920:::rbd_data.11b59d02cc38.0000000000000075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.6919889690000005, 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049089249999999997, 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 575 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312345+0000", 2024-09-16T12:59:27.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.353e-06 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312352+0000", 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7499999999999997e-06 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312370+0000", 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8210999999999999e-05 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312391+0000", 2024-09-16T12:59:27.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0968e-05 2024-09-16T12:59:27.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312965+0000", 2024-09-16T12:59:27.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057344399999999999 2024-09-16T12:59:27.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313134+0000", 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016979199999999999 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313322+0000", 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018742300000000001 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314160+0000", 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083821799999999995 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317190+0000", 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030301339999999999 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317219+0000", 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8368999999999999e-05 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317253+0000", 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4263e-05 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.037 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:591 4.19 4:9d8b1cf8:::rbd_data.11b59d02cc38.000000000000001f:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.338318+0000", 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.6660146359999999, 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003450617, 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 591 2024-09-16T12:59:27.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.338318+0000", 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.338318+0000", 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.338320+0000", 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.782e-06 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.338328+0000", 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7619999999999995e-06 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.338350+0000", 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2211000000000001e-05 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.338361+0000", 2024-09-16T12:59:27.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0560999999999999e-05 2024-09-16T12:59:27.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.338476+0000", 2024-09-16T12:59:27.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000114974 2024-09-16T12:59:27.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.338731+0000", 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025581300000000003 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.339330+0000", 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059893700000000002 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.341591+0000", 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022606670000000001 2024-09-16T12:59:27.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.341651+0000", 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0408000000000002e-05 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.341712+0000", 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0859000000000001e-05 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.341769+0000", 2024-09-16T12:59:27.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6643e-05 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:654 4.19 4:990dd591:::rbd_data.11b59d02cc38.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write 336970~8162] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.538529+0000", 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4658035629999997, 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033700520000000001, 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 654 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.538529+0000", 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.538529+0000", 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.538533+0000", 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3270000000000001e-06 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.538558+0000", 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5445000000000001e-05 2024-09-16T12:59:27.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.538603+0000", 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4999000000000001e-05 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.538618+0000", 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4980999999999999e-05 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.538726+0000", 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010834499999999999 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.539013+0000", 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028607399999999999 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.539314+0000", 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030153200000000003 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.541188+0000", 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018742100000000001 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.541745+0000", 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00055634599999999999 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.541834+0000", 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9058e-05 2024-09-16T12:59:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.541899+0000", 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5735000000000007e-05 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:718 4.11 4:882565b9:::rbd_data.11b59d02cc38.0000000000000008:head [set-alloc-hint object_size 1048576 write_size 1048576,write 655360~7987] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.708816+0000", 2024-09-16T12:59:27.050 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.2955166499999997, 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034197910000000002, 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 718 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.708816+0000", 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.708816+0000", 2024-09-16T12:59:27.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.708819+0000", 2024-09-16T12:59:27.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7240000000000001e-06 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.708841+0000", 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1577999999999999e-05 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.708892+0000", 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1598000000000003e-05 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.708903+0000", 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1209000000000001e-05 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709109+0000", 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020515 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709500+0000", 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039132899999999999 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709901+0000", 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040130400000000001 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.711815+0000", 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019139230000000001 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712083+0000", 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026778499999999998 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712172+0000", 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8842000000000001e-05 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712236+0000", 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4349000000000003e-05 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.057 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:933 4.1d 4:b8b88188:::rbd_data.11b59d02cc38.0000000000000021:head [set-alloc-hint object_size 1048576 write_size 1048576,write 616865~23473] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:27.057 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.155280+0000", 2024-09-16T12:59:27.057 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.8490531749999999, 2024-09-16T12:59:27.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033467369999999998, 2024-09-16T12:59:27.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 933 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.155280+0000", 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.155280+0000", 2024-09-16T12:59:27.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.155282+0000", 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.334e-06 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.155307+0000", 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4803999999999998e-05 2024-09-16T12:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.155336+0000", 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9047000000000002e-05 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.155347+0000", 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0988999999999999e-05 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.155501+0000", 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000154253 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.155756+0000", 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025478799999999999 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.156024+0000", 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000268423 2024-09-16T12:59:27.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.157800+0000", 2024-09-16T12:59:27.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001775446 2024-09-16T12:59:27.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.158460+0000", 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00066052799999999996 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.158552+0000", 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2088000000000003e-05 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.158627+0000", 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4036999999999996e-05 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.063 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:950 4.17 4:e9df2e13:::rbd_data.11b59d02cc38.0000000000000007:head [write 983040~36251] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.202396+0000", 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.8019372570000005, 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043336290000000003, 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 950 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202396+0000", 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202396+0000", 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202397+0000", 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7069999999999999e-06 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202433+0000", 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5688000000000001e-05 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202452+0000", 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8550999999999999e-05 2024-09-16T12:59:27.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202472+0000", 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0565999999999999e-05 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202586+0000", 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113328 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202802+0000", 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021687200000000001 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.203171+0000", 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036829700000000001 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.204781+0000", 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016104769999999999 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.206481+0000", 2024-09-16T12:59:27.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017001989999999999 2024-09-16T12:59:27.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.206620+0000", 2024-09-16T12:59:27.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139004 2024-09-16T12:59:27.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.206729+0000", 2024-09-16T12:59:27.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010894 2024-09-16T12:59:27.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.070 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.070 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:985 4.11 4:882565b9:::rbd_data.11b59d02cc38.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.304439+0000", 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6998941999999992, 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042693189999999997, 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 985 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304439+0000", 2024-09-16T12:59:27.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304439+0000", 2024-09-16T12:59:27.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304443+0000", 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-06 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304464+0000", 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0767999999999998e-05 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304526+0000", 2024-09-16T12:59:27.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1865999999999993e-05 2024-09-16T12:59:27.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304575+0000", 2024-09-16T12:59:27.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9178999999999998e-05 2024-09-16T12:59:27.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304715+0000", 2024-09-16T12:59:27.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014015699999999999 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304920+0000", 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020457299999999999 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.305215+0000", 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029488399999999998 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307142+0000", 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019277599999999999 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308529+0000", 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001386424 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308632+0000", 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010343700000000001 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308708+0000", 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5871000000000006e-05 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:994 4.1d 4:b8b88188:::rbd_data.11b59d02cc38.0000000000000021:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.304737+0000", 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6995954560000008, 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036374789999999999, 2024-09-16T12:59:27.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 994 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304737+0000", 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304737+0000", 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304739+0000", 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1799999999999999e-06 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304747+0000", 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.337e-06 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304767+0000", 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9504000000000002e-05 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304778+0000", 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1385e-05 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304912+0000", 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134556 2024-09-16T12:59:27.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.305129+0000", 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021696700000000001 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.305426+0000", 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029640100000000001 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307549+0000", 2024-09-16T12:59:27.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021235389999999998 2024-09-16T12:59:27.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308163+0000", 2024-09-16T12:59:27.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00061387799999999999 2024-09-16T12:59:27.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308286+0000", 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000122552 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308375+0000", 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9179999999999997e-05 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1001 4.1d 4:b8dd3499:::rbd_data.11b59d02cc38.0000000000000051:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.309177+0000", 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6951560059999995, 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049686330000000001, 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1001 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309177+0000", 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309177+0000", 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309179+0000", 2024-09-16T12:59:27.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1320000000000001e-06 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309187+0000", 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2479999999999996e-06 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309216+0000", 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8884e-05 2024-09-16T12:59:27.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309228+0000", 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1992e-05 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309365+0000", 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013697399999999999 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309849+0000", 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048348 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309985+0000", 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013675300000000001 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311601+0000", 2024-09-16T12:59:27.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001615472 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.314014+0000", 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024127950000000001 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.314102+0000", 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8324000000000003e-05 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.314146+0000", 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3578999999999997e-05 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1145 4.17 4:edd7f3ed:::image_client.0-clone2.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.733224+0000", 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2711085989999997, 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053216499999999998, 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T12:59:27.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1145 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733224+0000", 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733224+0000", 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733226+0000", 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0109999999999999e-06 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733234+0000", 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3440000000000003e-06 2024-09-16T12:59:27.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733255+0000", 2024-09-16T12:59:27.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1347000000000001e-05 2024-09-16T12:59:27.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733266+0000", 2024-09-16T12:59:27.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.117e-05 2024-09-16T12:59:27.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733330+0000", 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3577000000000005e-05 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733339+0000", 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0520000000000007e-06 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.738402+0000", 2024-09-16T12:59:27.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050630680000000004 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.738546+0000", 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014408100000000001 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.095 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1148 4.0 4:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.096 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.741509+0000", 2024-09-16T12:59:27.096 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2628240779999995, 2024-09-16T12:59:27.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055772950000000003, 2024-09-16T12:59:27.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1148 2024-09-16T12:59:27.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.097 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741509+0000", 2024-09-16T12:59:27.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741509+0000", 2024-09-16T12:59:27.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741511+0000", 2024-09-16T12:59:27.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8029999999999999e-06 2024-09-16T12:59:27.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741519+0000", 2024-09-16T12:59:27.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0779999999999996e-06 2024-09-16T12:59:27.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741541+0000", 2024-09-16T12:59:27.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2609999999999999e-05 2024-09-16T12:59:27.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741552+0000", 2024-09-16T12:59:27.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1001e-05 2024-09-16T12:59:27.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741622+0000", 2024-09-16T12:59:27.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.004e-05 2024-09-16T12:59:27.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741786+0000", 2024-09-16T12:59:27.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000163251 2024-09-16T12:59:27.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:27.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741966+0000", 2024-09-16T12:59:27.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018033400000000001 2024-09-16T12:59:27.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.742883+0000", 2024-09-16T12:59:27.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091699900000000005 2024-09-16T12:59:27.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.746880+0000", 2024-09-16T12:59:27.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039970769999999999 2024-09-16T12:59:27.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.747012+0000", 2024-09-16T12:59:27.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000131698 2024-09-16T12:59:27.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.747086+0000", 2024-09-16T12:59:27.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4404000000000001e-05 2024-09-16T12:59:27.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.105 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1256 4.19 4:9a6e2d6b:::rbd_data.11b53140b009.000000000000009c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 942080~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.105 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.032261+0000", 2024-09-16T12:59:27.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.9720721780000003, 2024-09-16T12:59:27.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003826454, 2024-09-16T12:59:27.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1256 2024-09-16T12:59:27.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.032261+0000", 2024-09-16T12:59:27.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.032261+0000", 2024-09-16T12:59:27.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.032262+0000", 2024-09-16T12:59:27.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1429999999999999e-06 2024-09-16T12:59:27.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.032294+0000", 2024-09-16T12:59:27.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1918000000000002e-05 2024-09-16T12:59:27.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.032311+0000", 2024-09-16T12:59:27.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7397999999999998e-05 2024-09-16T12:59:27.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.032328+0000", 2024-09-16T12:59:27.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6937000000000001e-05 2024-09-16T12:59:27.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.033471+0000", 2024-09-16T12:59:27.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011424390000000001 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.033694+0000", 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022336400000000001 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.034070+0000", 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000375819 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.035135+0000", 2024-09-16T12:59:27.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010653769999999999 2024-09-16T12:59:27.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.035982+0000", 2024-09-16T12:59:27.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084643299999999995 2024-09-16T12:59:27.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.036041+0000", 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9317000000000001e-05 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.036087+0000", 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6309e-05 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.113 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3482 4.19 4:9af53487:::rbd_data.11b5c378985b.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 3004685~21392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:21.932083+0000", 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0722497620000002, 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037447819999999999, 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3482 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.932083+0000", 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.932083+0000", 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.932085+0000", 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.932236+0000", 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015064899999999999 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.932259+0000", 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3604e-05 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.932268+0000", 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3569999999999998e-06 2024-09-16T12:59:27.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.932333+0000", 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4933999999999999e-05 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.932532+0000", 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000199333 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.932993+0000", 2024-09-16T12:59:27.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000460839 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.935022+0000", 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020295869999999998 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.935552+0000", 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000529898 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.935716+0000", 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016404399999999999 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.935828+0000", 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111611 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:3502 4.1b e41/31 4:dbfa2cf4:::rbd_data.11b5c378985b.0000000000000013:head v 41'8, mlcod=38'6)", 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:21.969284+0000", 2024-09-16T12:59:27.119 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0350490890000001, 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060809890000000002, 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.969284+0000", 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.969284+0000", 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.969286+0000", 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2079999999999999e-06 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971841+0000", 2024-09-16T12:59:27.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025551620000000001 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971871+0000", 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0230999999999999e-05 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971883+0000", 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1046000000000001e-05 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971961+0000", 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8560999999999995e-05 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.972001+0000", 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0204999999999999e-05 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.975320+0000", 2024-09-16T12:59:27.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033191800000000001 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.975365+0000", 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4396e-05 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3509 4.19 4:9af53487:::rbd_data.11b5c378985b.0000000000000015:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:27.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.007921+0000", 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.996411535, 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032908479999999999, 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3509 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007921+0000", 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007921+0000", 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007922+0000", 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6899999999999997e-07 2024-09-16T12:59:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007925+0000", 2024-09-16T12:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3670000000000001e-06 2024-09-16T12:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007934+0000", 2024-09-16T12:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4370000000000007e-06 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.007938+0000", 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3089999999999997e-06 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.008644+0000", 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000705523 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.008836+0000", 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019191400000000001 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.009109+0000", 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027346199999999998 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.010342+0000", 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012332980000000001 2024-09-16T12:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.011037+0000", 2024-09-16T12:59:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00069491099999999999 2024-09-16T12:59:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.011161+0000", 2024-09-16T12:59:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000124005 2024-09-16T12:59:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.011212+0000", 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0952999999999998e-05 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3536 4.14 4:2a646c7f:::rbd_data.11b5c378985b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.025341+0000", 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9789918740000001, 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067260820000000004, 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3536 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025341+0000", 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025341+0000", 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025343+0000", 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.739e-06 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025571+0000", 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022837000000000001 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025590+0000", 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8360999999999999e-05 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025597+0000", 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1910000000000003e-06 2024-09-16T12:59:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025667+0000", 2024-09-16T12:59:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0723999999999996e-05 2024-09-16T12:59:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025848+0000", 2024-09-16T12:59:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000180679 2024-09-16T12:59:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.027038+0000", 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001189854 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.031540+0000", 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045024829999999998 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.031922+0000", 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000381689 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.032005+0000", 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3141999999999998e-05 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.032067+0000", 2024-09-16T12:59:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1849999999999999e-05 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3616 4.14 4:2a646c7f:::rbd_data.11b5c378985b.0000000000000001:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.208182+0000", 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7961507570000004, 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037632210000000002, 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:27.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3616 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.208182+0000", 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:27.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.208182+0000", 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.208186+0000", 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000004e-06 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.208206+0000", 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9704999999999999e-05 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.208249+0000", 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3408000000000002e-05 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.208265+0000", 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5894000000000001e-05 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.208385+0000", 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012021800000000001 2024-09-16T12:59:27.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.208551+0000", 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016619399999999999 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.208795+0000", 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024306600000000001 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:27.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.210559+0000", 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001764027 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.211769+0000", 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001210652 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.211886+0000", 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116415 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:27.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:27.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:27.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.211945+0000", 2024-09-16T12:59:27.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9791999999999999e-05 2024-09-16T12:59:27.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.144 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:27.144 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:27.144 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:27.146 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:27.146 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone6 from 0x97e8ff0 (overlap 0x97e8ff0) to 0x88b818b 2024-09-16T12:59:27.146 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:27.135+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:27.223 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-16T12:59:27.226 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:27.226 INFO:teuthology.orchestra.run.smithi042.stdout:527 punch from 0x71713d9 to 0x717533d, (0x3f64 bytes) 2024-09-16T12:59:27.246 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:27.243+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:27.253 INFO:teuthology.orchestra.run.smithi042.stdout:529 writesame 0xced9398 thru 0xcee5880 data_size 0x9(0xc4e9 bytes) 2024-09-16T12:59:27.253 INFO:teuthology.orchestra.run.smithi042.stdout:530 write 0x76e3efa thru 0x76e7f93 (0x409a bytes) 2024-09-16T12:59:27.265 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:27.265 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:27.265 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:27.265 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:27.275 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:27.275 INFO:teuthology.orchestra.run.smithi042.stdout:532 read 0x3809a3c thru 0x38141ab (0xa770 bytes) 2024-09-16T12:59:27.276 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:27.277 INFO:teuthology.orchestra.run.smithi042.stdout:534 read 0xcb7e11a thru 0xcb817f8 (0x36df bytes) 2024-09-16T12:59:27.277 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:27.277 INFO:teuthology.orchestra.run.smithi042.stdout:536 write 0x25a8c02 thru 0x25b53b1 (0xc7b0 bytes) 2024-09-16T12:59:27.347 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:27.339+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:27.377 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:27.380 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:27.380 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:27.381 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:27.381 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:27.389 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:27.422 INFO:teuthology.orchestra.run.smithi042.stdout:537 punch from 0x6263796 to 0x6266092, (0x28fc bytes) 2024-09-16T12:59:27.422 INFO:teuthology.orchestra.run.smithi042.stdout:538 read 0xbac8e91 thru 0xbad7af4 (0xec64 bytes) 2024-09-16T12:59:27.422 INFO:teuthology.orchestra.run.smithi042.stdout:540 read 0x8fcc78e thru 0x8fcfcc5 (0x3538 bytes) 2024-09-16T12:59:27.423 INFO:teuthology.orchestra.run.smithi042.stdout:541 read 0x15daea9 thru 0x15ddcbf (0x2e17 bytes) 2024-09-16T12:59:27.424 INFO:teuthology.orchestra.run.smithi042.stdout:542 writesame 0x5750c39 thru 0x5755fbb data_size 0x1(0x5383 bytes) 2024-09-16T12:59:27.444 INFO:teuthology.orchestra.run.smithi042.stdout:543 writesame 0x8b0316c thru 0x8b09371 data_size 0x1(0x6206 bytes) 2024-09-16T12:59:27.447 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:27.443+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout:544 writesame 0x7962cd1 thru 0x796712a data_size 0xd(0x445a bytes) 2024-09-16T12:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout:545 read 0x81fe821 thru 0x8201cf8 (0x34d8 bytes) 2024-09-16T12:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout:547 punch from 0x922b32f to 0x9234f00, (0x9bd1 bytes) 2024-09-16T12:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout:548 writesame 0x8c00efa thru 0x8c0fc55 data_size 0x16(0xed5c bytes) 2024-09-16T12:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout:549 write 0x247f2f6 thru 0x2486c59 (0x7964 bytes) 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:7 4.1a 4:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:12.759313+0000", 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.725616905000001, 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074382090000000003, 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 7 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759313+0000", 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759313+0000", 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759316+0000", 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.213e-06 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759331+0000", 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5082999999999999e-05 2024-09-16T12:59:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759368+0000", 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6415999999999998e-05 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759384+0000", 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6039999999999999e-05 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759547+0000", 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163466 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.759563+0000", 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5262999999999999e-05 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.764413+0000", 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048502859999999997 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.764694+0000", 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028159099999999998 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.764933+0000", 2024-09-16T12:59:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023883300000000001 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.766322+0000", 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013884500000000001 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.766664+0000", 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034210000000000002 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.766717+0000", 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2932000000000001e-05 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:12.766751+0000", 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4536000000000003e-05 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:590 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.145660581, 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034258392999999998, 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 590 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339271+0000", 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339281+0000", 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6670000000000008e-06 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339307+0000", 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5907000000000001e-05 2024-09-16T12:59:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339317+0000", 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0356999999999999e-05 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339412+0000", 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4248999999999998e-05 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339533+0000", 2024-09-16T12:59:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012184900000000001 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.347489+0000", 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079559599999999998 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373423+0000", 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025933971 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373435+0000", 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1924e-05 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373492+0000", 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6842999999999998e-05 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373528+0000", 2024-09-16T12:59:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1248 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.027312+0000", 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4576181669999997, 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072161170000000002, 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1248 2024-09-16T12:59:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027312+0000", 2024-09-16T12:59:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027312+0000", 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027314+0000", 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.17e-06 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027325+0000", 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0910000000000001e-05 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027348+0000", 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2943e-05 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027369+0000", 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1002999999999999e-05 2024-09-16T12:59:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027435+0000", 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.656e-05 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027514+0000", 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8200000000000003e-05 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.027894+0000", 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038003800000000002 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033571+0000", 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056774640000000001 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034321+0000", 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074980800000000003 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034460+0000", 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138603 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034528+0000", 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8417999999999994e-05 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1250 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.028320+0000", 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4566098249999992, 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086382009999999999, 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1250 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028320+0000", 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028320+0000", 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028323+0000", 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4600000000000002e-06 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028333+0000", 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0813e-05 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028355+0000", 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1535999999999998e-05 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028366+0000", 2024-09-16T12:59:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1389999999999999e-05 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029349+0000", 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00098287900000000009 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029535+0000", 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018608000000000001 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030223+0000", 2024-09-16T12:59:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068792300000000001 2024-09-16T12:59:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036722+0000", 2024-09-16T12:59:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006498658 2024-09-16T12:59:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036805+0000", 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2880999999999995e-05 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036892+0000", 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7665999999999997e-05 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036958+0000", 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5914999999999995e-05 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1251 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.028790+0000", 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4561397249999999, 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083182099999999995, 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1251 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028790+0000", 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028790+0000", 2024-09-16T12:59:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028792+0000", 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.787e-06 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028801+0000", 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3819999999999993e-06 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028819+0000", 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7237000000000001e-05 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028830+0000", 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1749999999999999e-05 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.031172+0000", 2024-09-16T12:59:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002341409 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.031361+0000", 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018938900000000001 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032087+0000", 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072580400000000003 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036792+0000", 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004705234 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036974+0000", 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018133099999999999 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.037043+0000", 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9115999999999994e-05 2024-09-16T12:59:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.037109+0000", 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5771000000000004e-05 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1252 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.029328+0000", 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4556015529999993, 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096787239999999997, 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1252 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029328+0000", 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029328+0000", 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029330+0000", 2024-09-16T12:59:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.73e-06 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029337+0000", 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0099999999999998e-06 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029355+0000", 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7391999999999999e-05 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029363+0000", 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8200000000000003e-06 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033606+0000", 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042427139999999999 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033658+0000", 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1694999999999999e-05 2024-09-16T12:59:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033904+0000", 2024-09-16T12:59:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024602000000000001 2024-09-16T12:59:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.038868+0000", 2024-09-16T12:59:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049644110000000002 2024-09-16T12:59:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.038883+0000", 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.45e-05 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.038950+0000", 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7372999999999993e-05 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.039007+0000", 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7058999999999997e-05 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1253 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.029790+0000", 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4551396200000006, 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099550560000000003, 2024-09-16T12:59:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1253 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029790+0000", 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029790+0000", 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029791+0000", 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.083e-06 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029796+0000", 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.994e-06 2024-09-16T12:59:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029807+0000", 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0502999999999999e-05 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029822+0000", 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5412000000000001e-05 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034742+0000", 2024-09-16T12:59:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049191699999999996 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034901+0000", 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015901399999999999 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035563+0000", 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066223200000000001 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.039207+0000", 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036439139999999998 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.039687+0000", 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047999400000000003 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.039721+0000", 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4650000000000002e-05 2024-09-16T12:59:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.039745+0000", 2024-09-16T12:59:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4090000000000001e-05 2024-09-16T12:59:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1263 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.044925+0000", 2024-09-16T12:59:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4400052750000008, 2024-09-16T12:59:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070913340000000004, 2024-09-16T12:59:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1263 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.044925+0000", 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.044925+0000", 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.044926+0000", 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.316e-06 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.044932+0000", 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7100000000000004e-06 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.044944+0000", 2024-09-16T12:59:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2099999999999999e-05 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.044951+0000", 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7719999999999997e-06 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.046595+0000", 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016447300000000001 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.046726+0000", 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130338 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047085+0000", 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035967600000000002 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051489+0000", 2024-09-16T12:59:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044039739999999997 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051895+0000", 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040589199999999998 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051963+0000", 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7384000000000004e-05 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.052016+0000", 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3442000000000001e-05 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1265 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.045449+0000", 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4394811720000007, 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072166940000000001, 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1265 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045449+0000", 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045449+0000", 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045450+0000", 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.11e-06 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045470+0000", 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0171000000000001e-05 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045484+0000", 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4089e-05 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045492+0000", 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2790000000000001e-06 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.048366+0000", 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028741679999999999 2024-09-16T12:59:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.048479+0000", 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113202 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.048889+0000", 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041052500000000002 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051527+0000", 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026380129999999998 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.052567+0000", 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001039136 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.052624+0000", 2024-09-16T12:59:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.736e-05 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.052666+0000", 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1641000000000003e-05 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1266 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.045696+0000", 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4392336740000005, 2024-09-16T12:59:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072059050000000003, 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1266 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045696+0000", 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045696+0000", 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045697+0000", 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7699999999999992e-07 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045706+0000", 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4800000000000001e-06 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045714+0000", 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2819999999999996e-06 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.045718+0000", 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1760000000000003e-06 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.049314+0000", 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035960670000000001 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.049431+0000", 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116236 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.049856+0000", 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042557199999999998 2024-09-16T12:59:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.052213+0000", 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023569960000000001 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.052775+0000", 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056204300000000003 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.052828+0000", 2024-09-16T12:59:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3300000000000001e-05 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.052902+0000", 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3776000000000007e-05 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1296 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.075871+0000", 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4090595080000003, 2024-09-16T12:59:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098124590000000008, 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1296 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.075871+0000", 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.075871+0000", 2024-09-16T12:59:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.075871+0000", 2024-09-16T12:59:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0699999999999996e-07 2024-09-16T12:59:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.075875+0000", 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.856e-06 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.075883+0000", 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7109999999999995e-06 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.075887+0000", 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.16e-06 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.078244+0000", 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002356988 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.078387+0000", 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014304500000000001 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079004+0000", 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061665600000000004 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.085590+0000", 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065865610000000003 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.085626+0000", 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5528999999999998e-05 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.085660+0000", 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3776000000000003e-05 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.085683+0000", 2024-09-16T12:59:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3470000000000001e-05 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1297 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.076257+0000", 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4086733210000002, 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010539341000000001, 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1297 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076257+0000", 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076257+0000", 2024-09-16T12:59:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076258+0000", 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4980000000000001e-06 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076267+0000", 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7399999999999993e-06 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076282+0000", 2024-09-16T12:59:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4701e-05 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076289+0000", 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999997e-06 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080321+0000", 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040322689999999998 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080472+0000", 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015052800000000001 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081144+0000", 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067246999999999999 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086474+0000", 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053297589999999999 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086708+0000", 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000234131 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086759+0000", 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0679e-05 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086796+0000", 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7490000000000002e-05 2024-09-16T12:59:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1298 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.076549+0000", 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4083813410000001, 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012581233000000001, 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1298 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076549+0000", 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076549+0000", 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076550+0000", 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9599999999999998e-07 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076554+0000", 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8289999999999998e-06 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076565+0000", 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0446e-05 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076569+0000", 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5220000000000001e-06 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.082476+0000", 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059068799999999998 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.082603+0000", 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012666399999999999 2024-09-16T12:59:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083102+0000", 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049894900000000005 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.087350+0000", 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042485379999999996 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.088968+0000", 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001617804 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089056+0000", 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7690999999999998e-05 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089130+0000", 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4014e-05 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1299 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.076902+0000", 2024-09-16T12:59:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4080279789999999, 2024-09-16T12:59:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012319369, 2024-09-16T12:59:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1299 2024-09-16T12:59:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076902+0000", 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076902+0000", 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076903+0000", 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.22e-06 2024-09-16T12:59:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076910+0000", 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6080000000000001e-06 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076931+0000", 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1461e-05 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076938+0000", 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8000000000000001e-06 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083946+0000", 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070081229999999998 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084053+0000", 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107092 2024-09-16T12:59:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084240+0000", 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018616800000000001 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.087368+0000", 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031283790000000001 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089142+0000", 2024-09-16T12:59:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00177454 2024-09-16T12:59:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089184+0000", 2024-09-16T12:59:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1641000000000003e-05 2024-09-16T12:59:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089221+0000", 2024-09-16T12:59:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7336999999999998e-05 2024-09-16T12:59:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1300 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.077228+0000", 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4077017359999999, 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012068957999999999, 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1300 2024-09-16T12:59:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.077228+0000", 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.077228+0000", 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.077229+0000", 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0610000000000001e-06 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.077235+0000", 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8690000000000001e-06 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.077249+0000", 2024-09-16T12:59:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3291e-05 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.077254+0000", 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7679999999999997e-06 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084564+0000", 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073095540000000002 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084710+0000", 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014609399999999999 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.085146+0000", 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043558399999999998 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.087407+0000", 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022618859999999998 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089230+0000", 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001822801 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089264+0000", 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3390999999999999e-05 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089297+0000", 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3658999999999997e-05 2024-09-16T12:59:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:3502 4.1b 4:dbfa2cf4:::rbd_data.11b5c378985b.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:21.967018+0000", 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.5179115469999998, 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01113135, 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3502 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.967018+0000", 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.967018+0000", 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.967019+0000", 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4499999999999995e-07 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.970614+0000", 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00359473 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.970635+0000", 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0965999999999999e-05 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.970642+0000", 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3749999999999997e-06 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.970717+0000", 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4411000000000003e-05 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.970890+0000", 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017349499999999999 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.971262+0000", 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037121399999999997 2024-09-16T12:59:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.976238+0000", 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049768030000000001 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.978037+0000", 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00179904 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.978101+0000", 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3761000000000002e-05 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:21.978150+0000", 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8609999999999997e-05 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5052 4.1a 4:5ac70f08:::rbd_data.11b517fb51dd.0000000000000003:9 [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275732+0000", 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20919805499999999, 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071291369999999998, 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5052 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275732+0000", 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275732+0000", 2024-09-16T12:59:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275733+0000", 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2449999999999999e-06 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275739+0000", 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6289999999999998e-06 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275756+0000", 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7130999999999999e-05 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275766+0000", 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6770000000000007e-06 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282496+0000", 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006730649 2024-09-16T12:59:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282681+0000", 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018432 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282861+0000", 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018048599999999999 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5054 4.1a 4:5b456d59:::rbd_data.11b517fb51dd.0000000000000000:9 [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275776+0000", 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20915371899999999, 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078603300000000004, 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5054 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275776+0000", 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275776+0000", 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275777+0000", 2024-09-16T12:59:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4300000000000001e-07 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275782+0000", 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9649999999999999e-06 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275801+0000", 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9046e-05 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275811+0000", 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2909999999999996e-06 2024-09-16T12:59:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.283152+0000", 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073418609999999999 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.283452+0000", 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029928499999999999 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.283637+0000", 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018493900000000001 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5059 4.1a 4:5ac70f08:::rbd_data.11b517fb51dd.0000000000000003:9 [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275821+0000", 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20910904899999999, 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083722250000000005, 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5059 2024-09-16T12:59:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275821+0000", 2024-09-16T12:59:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275821+0000", 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275822+0000", 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0920000000000001e-06 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275827+0000", 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7999999999999998e-06 2024-09-16T12:59:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275843+0000", 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5682e-05 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275852+0000", 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4399999999999994e-06 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.283773+0000", 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079211639999999996 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.283975+0000", 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020210200000000001 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.284193+0000", 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021794499999999999 2024-09-16T12:59:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5063 4.13 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22711298~51120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.17551578300000001, 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111447293, 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5063 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309416+0000", 2024-09-16T12:59:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T12:59:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358861+0000", 2024-09-16T12:59:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049444717999999999 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358893+0000", 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2279e-05 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358902+0000", 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4390000000000006e-06 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359062+0000", 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016014100000000001 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359330+0000", 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026734699999999997 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.362566+0000", 2024-09-16T12:59:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032358769999999999 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.417541+0000", 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054975345000000002 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420706+0000", 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031651380000000001 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420790+0000", 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4283999999999995e-05 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420862+0000", 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1138999999999998e-05 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:27.620 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:27.622 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:27.622 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:27.615+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:27.623 INFO:teuthology.orchestra.run.smithi042.stdout:552 trunc from 0xcee5881 to 0x5e86ff0 2024-09-16T12:59:27.623 INFO:teuthology.orchestra.run.smithi042.stdout:553 read 0x326e919 thru 0x327a169 (0xb851 bytes) 2024-09-16T12:59:27.623 INFO:teuthology.orchestra.run.smithi042.stdout:555 read 0x32d5902 thru 0x32e2766 (0xce65 bytes) 2024-09-16T12:59:27.623 INFO:teuthology.orchestra.run.smithi042.stdout:556 write 0x6ded0cd thru 0x6dee04c (0xf80 bytes) 2024-09-16T12:59:27.623 INFO:teuthology.orchestra.run.smithi042.stdout:557 writesame 0xc917b08 thru 0xc9274f6 data_size 0x6d(0xf9ef bytes) 2024-09-16T12:59:27.623 INFO:teuthology.orchestra.run.smithi042.stdout:558 punch from 0x52a6e49 to 0x52a7af0, (0xca7 bytes) 2024-09-16T12:59:27.623 INFO:teuthology.orchestra.run.smithi042.stdout:560 read 0x79ae9f6 thru 0x79b45c6 (0x5bd1 bytes) 2024-09-16T12:59:27.623 INFO:teuthology.orchestra.run.smithi042.stdout:561 write 0x73b1f32 thru 0x73c122f (0xf2fe bytes) 2024-09-16T12:59:27.624 INFO:teuthology.orchestra.run.smithi042.stdout:562 trunc from 0xc9274f7 to 0x5f82375 2024-09-16T12:59:27.624 INFO:teuthology.orchestra.run.smithi042.stdout:563 writesame 0x28d944d thru 0x28dc33e data_size 0x3(0x2ef2 bytes) 2024-09-16T12:59:27.624 INFO:teuthology.orchestra.run.smithi042.stdout:564 trunc from 0x5f82375 to 0x990935f 2024-09-16T12:59:27.656 INFO:teuthology.orchestra.run.smithi042.stdout:566 write 0xcd2a301 thru 0xcd31891 (0x7591 bytes) 2024-09-16T12:59:27.677 INFO:teuthology.orchestra.run.smithi042.stdout:568 trunc from 0xcd31892 to 0x780ccf3 2024-09-16T12:59:27.702 INFO:teuthology.orchestra.run.smithi042.stdout:569 punch from 0x7431a13 to 0x744144f, (0xfa3c bytes) 2024-09-16T12:59:27.702 INFO:teuthology.orchestra.run.smithi042.stdout:570 read 0x72196e3 thru 0x721cf19 (0x3837 bytes) 2024-09-16T12:59:27.702 INFO:teuthology.orchestra.run.smithi042.stdout:571 punch from 0x714f1e7 to 0x715059b, (0x13b4 bytes) 2024-09-16T12:59:27.703 INFO:teuthology.orchestra.run.smithi042.stdout:572 write 0x41a81aa thru 0x41ae590 (0x63e7 bytes) 2024-09-16T12:59:27.722 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:27.711+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:27.729 INFO:teuthology.orchestra.run.smithi042.stdout:573 writesame 0xb80221d thru 0xb804cf8 data_size 0xd(0x2adc bytes) 2024-09-16T12:59:27.741 INFO:teuthology.orchestra.run.smithi042.stdout:574 trunc from 0xb804cf9 to 0x61e9cff 2024-09-16T12:59:27.755 INFO:teuthology.orchestra.run.smithi042.stdout:577 writesame 0x54ea61a thru 0x54f98ce data_size 0x8b(0xf2b5 bytes) 2024-09-16T12:59:27.758 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:27.758 INFO:teuthology.orchestra.run.smithi042.stdout:579 read 0x1f651c4 thru 0x1f66de0 (0x1c1d bytes) 2024-09-16T12:59:27.775 INFO:teuthology.orchestra.run.smithi042.stdout:580 write 0x8668e34 thru 0x8670484 (0x7651 bytes) 2024-09-16T12:59:27.779 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:27.780 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:27.780 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:27.780 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:27.781 INFO:teuthology.orchestra.run.smithi042.stdout:582 read 0x7161687 thru 0x7167ff4 (0x696e bytes) 2024-09-16T12:59:27.792 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:27.795 INFO:teuthology.orchestra.run.smithi042.stdout:583 trunc from 0x8670485 to 0xe096280 2024-09-16T12:59:27.805 INFO:teuthology.orchestra.run.smithi042.stdout:585 writesame 0x4fe0d3b thru 0x4fe987f data_size 0x1(0x8b45 bytes) 2024-09-16T12:59:27.833 INFO:teuthology.orchestra.run.smithi042.stdout:586 write 0x918eca8 thru 0x9196e11 (0x816a bytes) 2024-09-16T12:59:27.838 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:27.835+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:27.839 INFO:teuthology.orchestra.run.smithi042.stdout:587 write 0x21e7159 thru 0x21eaa12 (0x38ba bytes) 2024-09-16T12:59:27.840 INFO:teuthology.orchestra.run.smithi042.stdout:588 punch from 0x991c898 to 0x9926c75, (0xa3dd bytes) 2024-09-16T12:59:27.841 INFO:teuthology.orchestra.run.smithi042.stdout:589 writesame 0x7055669 thru 0x7064a3f data_size 0x1(0xf3d7 bytes) 2024-09-16T12:59:27.876 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T12:59:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T12:59:27.890 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:27.890 INFO:teuthology.orchestra.run.smithi042.stdout:592 writesame 0x58de7d3 thru 0x58e469b data_size 0x17(0x5ec9 bytes) 2024-09-16T12:59:27.895 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:27.895 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:27.895 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:27.895 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:27.896 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:27.896 INFO:teuthology.orchestra.run.smithi042.stdout:594 punch from 0x7472d90 to 0x747c854, (0x9ac4 bytes) 2024-09-16T12:59:27.897 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T12:59:27.897 INFO:teuthology.orchestra.run.smithi042.stdout:595 write 0xbbe6d8d thru 0xbbf4a46 (0xdcba bytes) 2024-09-16T12:59:27.900 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:27.901 INFO:teuthology.orchestra.run.smithi042.stdout:597 write 0xa8d0609 thru 0xa8d76d9 (0x70d1 bytes) 2024-09-16T12:59:27.906 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:27.907 INFO:teuthology.orchestra.run.smithi042.stdout:598 write 0xc9e0db7 thru 0xc9ecbf9 (0xbe43 bytes) 2024-09-16T12:59:27.913 INFO:teuthology.orchestra.run.smithi042.stdout:599 writesame 0xdc110de thru 0xdc16482 data_size 0x31(0x53a5 bytes) 2024-09-16T12:59:27.924 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:27.915+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:27.931 INFO:teuthology.orchestra.run.smithi042.stdout:600 trunc from 0xe096280 to 0x720ff9d 2024-09-16T12:59:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:601 trunc from 0x720ff9d to 0xae0a0b7 2024-09-16T12:59:27.971 INFO:teuthology.orchestra.run.smithi042.stdout:602 punch from 0x2db71a7 to 0x2dbeea0, (0x7cf9 bytes) 2024-09-16T12:59:27.988 INFO:teuthology.orchestra.run.smithi042.stdout:603 writesame 0xea5fbfd thru 0xea6d7a8 data_size 0x22(0xdbac bytes) 2024-09-16T12:59:27.989 INFO:teuthology.orchestra.run.smithi042.stdout:605 read 0x5ecd6d4 thru 0x5ed5bd1 (0x84fe bytes) 2024-09-16T12:59:27.994 INFO:teuthology.orchestra.run.smithi042.stdout:607 trunc from 0xea6d7a9 to 0xb109867 2024-09-16T12:59:28.003 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T12:59:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.368836799999997, 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006297965, 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629315+0000", 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7080000000000002e-06 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629324+0000", 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0250000000000008e-06 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629340+0000", 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.59e-05 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629351+0000", 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1236999999999999e-05 2024-09-16T12:59:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629565+0000", 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000213482 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629582+0000", 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7482999999999998e-05 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.631980+0000", 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023977830000000001 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.632247+0000", 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000266867 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.632356+0000", 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010894 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.633744+0000", 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013876240000000001 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635507+0000", 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001763288 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635562+0000", 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5096999999999998e-05 2024-09-16T12:59:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635608+0000", 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5531000000000002e-05 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.356771203000001, 2024-09-16T12:59:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063927960000000001, 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641378+0000", 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6429999999999999e-06 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641390+0000", 2024-09-16T12:59:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1912e-05 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641422+0000", 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2666999999999997e-05 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641448+0000", 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5636e-05 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641623+0000", 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017479299999999999 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641641+0000", 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8369999999999999e-05 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.644629+0000", 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00298745 2024-09-16T12:59:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.644995+0000", 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000366688 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.645348+0000", 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035283200000000003 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647159+0000", 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018104760000000001 2024-09-16T12:59:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647631+0000", 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047253799999999999 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647708+0000", 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6715e-05 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647768+0000", 2024-09-16T12:59:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0075999999999999e-05 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.349191243999996, 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006003988, 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 7 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648958+0000", 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0280000000000001e-06 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648970+0000", 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1374e-05 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649002+0000", 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2298999999999997e-05 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649014+0000", 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2442e-05 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649099+0000", 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4655999999999997e-05 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649107+0000", 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0800000000000006e-06 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.652487+0000", 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003380241 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.652733+0000", 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024601000000000001 2024-09-16T12:59:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.653091+0000", 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035755599999999998 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654603+0000", 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001512163 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654795+0000", 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000192179 2024-09-16T12:59:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654875+0000", 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0211999999999997e-05 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654959+0000", 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3747999999999998e-05 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:11 4.1f e34/31 4:fa654f18:::rbd_id.image_client.0:head v 34'1, mlcod=0'0)", 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:13.789305+0000", 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.208841659000001, 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051335770000000003, 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789305+0000", 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789305+0000", 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789306+0000", 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1850000000000001e-06 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789317+0000", 2024-09-16T12:59:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1344e-05 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789348+0000", 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0545e-05 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789363+0000", 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.485e-05 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789438+0000", 2024-09-16T12:59:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4884999999999999e-05 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789482+0000", 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4731999999999998e-05 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794352+0000", 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048695079999999998 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794438+0000", 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6527999999999996e-05 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:439 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.005732+0000", 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.992414073999999, 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050891779999999998, 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 439 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005732+0000", 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005732+0000", 2024-09-16T12:59:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005734+0000", 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4139999999999999e-06 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005741+0000", 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.503e-06 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005768+0000", 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6982e-05 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005787+0000", 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8974000000000002e-05 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.007568+0000", 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017811070000000001 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.007851+0000", 2024-09-16T12:59:28.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028264799999999998 2024-09-16T12:59:28.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.008426+0000", 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057548500000000004 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.009913+0000", 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001486376 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.010615+0000", 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070175700000000001 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.010750+0000", 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135685 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.010822+0000", 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1247000000000004e-05 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:513 4.8 4:132f6b09:::rbd_data.11b59d02cc38.000000000000008c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.817053883, 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054902010000000001, 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 513 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181095+0000", 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4059999999999999e-06 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181106+0000", 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0868e-05 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181145+0000", 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9168000000000002e-05 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181157+0000", 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2343e-05 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181229+0000", 2024-09-16T12:59:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1264000000000001e-05 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181415+0000", 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000186804 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181722+0000", 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030624200000000003 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.185844+0000", 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041219359999999997 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186379+0000", 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053546599999999996 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186495+0000", 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115575 2024-09-16T12:59:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186583+0000", 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8128999999999996e-05 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:532 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.219756+0000", 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.778390718000001, 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051249149999999999, 2024-09-16T12:59:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 532 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219756+0000", 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219756+0000", 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219758+0000", 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3099999999999999e-06 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219770+0000", 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1742999999999999e-05 2024-09-16T12:59:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219805+0000", 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5102999999999999e-05 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219818+0000", 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219971+0000", 2024-09-16T12:59:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015284000000000001 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.220132+0000", 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000161905 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.220456+0000", 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032376300000000002 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.221604+0000", 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001148135 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.224727+0000", 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003122545 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.224827+0000", 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9901000000000005e-05 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.224881+0000", 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3786999999999998e-05 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:542 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.764114261, 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060375849999999998, 2024-09-16T12:59:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 542 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234033+0000", 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8599999999999996e-07 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234038+0000", 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4830000000000001e-06 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234047+0000", 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3109999999999995e-06 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234053+0000", 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6339999999999998e-06 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.235355+0000", 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013027589999999999 2024-09-16T12:59:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.235617+0000", 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026182000000000001 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.236547+0000", 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093014599999999997 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239456+0000", 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029083289999999999 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239788+0000", 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033232900000000002 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239946+0000", 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015829700000000001 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.240070+0000", 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012349099999999999 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:571 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.689082371, 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005551762, 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 571 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309066+0000", 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2060000000000001e-06 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309080+0000", 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3413e-05 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309114+0000", 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4180000000000001e-05 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309126+0000", 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2129e-05 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309312+0000", 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018575800000000001 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309477+0000", 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000164885 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.310054+0000", 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057696199999999998 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.311809+0000", 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001755047 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314482+0000", 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026735330000000001 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314559+0000", 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6884000000000005e-05 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314616+0000", 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6765000000000003e-05 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:589 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.335679+0000", 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.662467653, 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050963889999999998, 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 589 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335679+0000", 2024-09-16T12:59:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335679+0000", 2024-09-16T12:59:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335680+0000", 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5700000000000001e-07 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335684+0000", 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6149999999999999e-06 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335697+0000", 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2465e-05 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335703+0000", 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7870000000000004e-06 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.336921+0000", 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001217625 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.337168+0000", 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024640300000000002 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.337796+0000", 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00062875099999999998 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340375+0000", 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025782579999999999 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340644+0000", 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026920399999999999 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340712+0000", 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639000000000005e-05 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340775+0000", 2024-09-16T12:59:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2785000000000003e-05 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:638 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.494910808, 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052595139999999999, 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 638 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503237+0000", 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.02e-06 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503248+0000", 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1114000000000001e-05 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503263+0000", 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092999999999999e-05 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503273+0000", 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0413e-05 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.504835+0000", 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001562164 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.505031+0000", 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000195465 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.505731+0000", 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070030899999999998 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507931+0000", 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022001429999999999 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508299+0000", 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036765399999999997 2024-09-16T12:59:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508405+0000", 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106368 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508495+0000", 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9771000000000005e-05 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:746 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.229011063, 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052549399999999996, 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 746 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769137+0000", 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330000000000001e-06 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769144+0000", 2024-09-16T12:59:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2660000000000004e-06 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769162+0000", 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7997000000000001e-05 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769178+0000", 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6305999999999999e-05 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.770704+0000", 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015260880000000001 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.770965+0000", 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026027800000000002 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.771962+0000", 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099762399999999991 2024-09-16T12:59:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.773232+0000", 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012702499999999999 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774136+0000", 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000903355 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774275+0000", 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139431 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774390+0000", 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011511200000000001 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:821 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.080098827, 2024-09-16T12:59:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052264670000000003, 2024-09-16T12:59:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.078 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 821 2024-09-16T12:59:28.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.078 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:28.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918049+0000", 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.734e-06 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918058+0000", 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918078+0000", 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0163e-05 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918100+0000", 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1396999999999999e-05 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.919700+0000", 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001600838 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.919878+0000", 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177276 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.920899+0000", 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010218219999999999 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.922776+0000", 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001876436 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923007+0000", 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023069799999999999 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923160+0000", 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015317299999999999 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923274+0000", 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114298 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:843 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.961705+0000", 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.036441183999999, 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049882310000000001, 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 843 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961705+0000", 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961705+0000", 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961706+0000", 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6200000000000006e-07 2024-09-16T12:59:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961710+0000", 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8619999999999996e-06 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961718+0000", 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7999999999999999e-06 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961723+0000", 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6859999999999997e-06 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.963536+0000", 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018133240000000001 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.963693+0000", 2024-09-16T12:59:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015719 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.964589+0000", 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089570199999999996 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.965980+0000", 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001391572 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.966585+0000", 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060418400000000001 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.966646+0000", 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1038999999999996e-05 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.966693+0000", 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.791e-05 2024-09-16T12:59:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1076 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.433166+0000", 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5649807090000003, 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051147570000000002, 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1076 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433166+0000", 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433166+0000", 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433168+0000", 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.57e-06 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433180+0000", 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2206e-05 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433215+0000", 2024-09-16T12:59:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4106000000000002e-05 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433228+0000", 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3159999999999999e-05 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433369+0000", 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014099699999999999 2024-09-16T12:59:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433647+0000", 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027864699999999998 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.434793+0000", 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011460559999999999 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.437959+0000", 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031659980000000002 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.438128+0000", 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000168908 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.438232+0000", 2024-09-16T12:59:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103426 2024-09-16T12:59:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.438280+0000", 2024-09-16T12:59:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8683000000000001e-05 2024-09-16T12:59:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1077 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5648971340000006, 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062511429999999998, 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1077 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433250+0000", 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1370000000000001e-06 2024-09-16T12:59:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433256+0000", 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433266+0000", 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0529e-05 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433278+0000", 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2363e-05 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.435866+0000", 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025878519999999999 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.436125+0000", 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002587 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.436955+0000", 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082961500000000002 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.438686+0000", 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017308930000000001 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439352+0000", 2024-09-16T12:59:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066689400000000002 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439455+0000", 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102876 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439500+0000", 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5151000000000002e-05 2024-09-16T12:59:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:3528 4.8 4:15914a01:::rbd_data.11b5c378985b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.9802392050000002, 2024-09-16T12:59:28.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051798410000000001, 2024-09-16T12:59:28.104 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.104 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:28.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.105 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3528 2024-09-16T12:59:28.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.105 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:28.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:28.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017909+0000", 2024-09-16T12:59:28.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.283e-06 2024-09-16T12:59:28.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017915+0000", 2024-09-16T12:59:28.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4729999999999997e-06 2024-09-16T12:59:28.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017941+0000", 2024-09-16T12:59:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5910000000000001e-05 2024-09-16T12:59:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017951+0000", 2024-09-16T12:59:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0557000000000001e-05 2024-09-16T12:59:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.018552+0000", 2024-09-16T12:59:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060063100000000002 2024-09-16T12:59:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.018663+0000", 2024-09-16T12:59:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011089199999999999 2024-09-16T12:59:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.023087+0000", 2024-09-16T12:59:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044240950000000003 2024-09-16T12:59:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:4787 4.15 4:a80ea28d:::rbd_header.11b5b1b9bb41:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.127950886, 2024-09-16T12:59:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060070260000000004, 2024-09-16T12:59:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4787 2024-09-16T12:59:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870199+0000", 2024-09-16T12:59:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2660000000000002e-06 2024-09-16T12:59:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870212+0000", 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2794e-05 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870248+0000", 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6798999999999998e-05 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870266+0000", 2024-09-16T12:59:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7288000000000001e-05 2024-09-16T12:59:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870385+0000", 2024-09-16T12:59:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119671 2024-09-16T12:59:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870394+0000", 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4300000000000006e-06 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.873802+0000", 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034079269999999998 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.873905+0000", 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102895 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.874314+0000", 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040942099999999998 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.875454+0000", 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001139784 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876028+0000", 2024-09-16T12:59:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057454900000000004 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876132+0000", 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010318399999999999 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876203+0000", 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1018000000000003e-05 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5060 4.7 4:e6558db2:::rbd_data.11b517fb51dd.0000000000000002:9 [sparse-read 2097152~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.72254611800000001, 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074022819999999996, 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5060 2024-09-16T12:59:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275602+0000", 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1509999999999998e-06 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275611+0000", 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0490000000000004e-06 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275632+0000", 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1115999999999999e-05 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275643+0000", 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1394e-05 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.279273+0000", 2024-09-16T12:59:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003629546 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.279360+0000", 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7213000000000007e-05 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.283003+0000", 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003642813 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5061 4.7 4:e6558db2:::rbd_data.11b517fb51dd.0000000000000002:9 [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.72249308999999995, 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073381729999999999, 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5061 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275654+0000", 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.074e-06 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275658+0000", 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6880000000000001e-06 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275668+0000", 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7839999999999998e-06 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275674+0000", 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9889999999999997e-06 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282791+0000", 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071176089999999996 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282884+0000", 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2085999999999999e-05 2024-09-16T12:59:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282992+0000", 2024-09-16T12:59:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010794300000000001 2024-09-16T12:59:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.132 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:28.135 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout:609 write 0xafa13bb thru 0xafa66e5 (0x532b bytes) 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:611 punch from 0x909a78f to 0x90a6743, (0xbfb4 bytes) 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:613 read 0x4f6bf2e thru 0x4f779ff (0xbad2 bytes) 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:614 punch from 0x1a50a5d to 0x1a56538, (0x5adb bytes) 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:615 trunc from 0xb109867 to 0xcc0f8be 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:618 write 0x115cc6a thru 0x11606c5 (0x3a5c bytes) 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:619 write 0x3d90a36 thru 0x3d92132 (0x16fd bytes) 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:620 punch from 0x3d5c8d9 to 0x3d6401c, (0x7743 bytes) 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:621 punch from 0x6418c9a to 0x641bbf9, (0x2f5f bytes) 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:624 write 0x99327d1 thru 0x993a7d1 (0x8001 bytes) 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:625 read 0x1ad15d2 thru 0x1adf306 (0xdd35 bytes) 2024-09-16T12:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout:627 read 0x8d90bd7 thru 0x8d952e1 (0x470b bytes) 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:628 write 0x394073a thru 0x394521e (0x4ae5 bytes) 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:629 writesame 0xbd320ae thru 0xbd4119d data_size 0x78(0xf0f0 bytes) 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:631 read 0x5511439 thru 0x551a3e1 (0x8fa9 bytes) 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:632 write 0x5e3c3d8 thru 0x5e43e02 (0x7a2b bytes) 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:633 write 0x2084452 thru 0x208ddc8 (0x9977 bytes) 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:635 read 0xca02570 thru 0xca103d6 (0xde67 bytes) 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:636 trunc from 0xcc0f8be to 0xcef145f 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:637 trunc from 0xcef145f to 0xc193f41 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:638 write 0x3509c2f thru 0x350eaff (0x4ed1 bytes) 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:641 write 0x1ecec0a thru 0x1ed145b (0x2852 bytes) 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:642 write 0xf2acbb thru 0xf324a0 (0x77e6 bytes) 2024-09-16T12:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout:644 write 0xd3da066 thru 0xd3e48a9 (0xa844 bytes) 2024-09-16T12:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout:648 write 0xb70085 thru 0xb746af (0x462b bytes) 2024-09-16T12:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout:650 write 0x626029b thru 0x626488b (0x45f1 bytes) 2024-09-16T12:59:28.140 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:28.123+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout:653 trunc from 0xd3e48aa to 0x77f687e 2024-09-16T12:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout:654 writesame 0xbbe468 thru 0xbcc8c0 data_size 0x7(0xe459 bytes) 2024-09-16T12:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout:655 punch from 0x68d5350 to 0x68df18d, (0x9e3d bytes) 2024-09-16T12:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout:659 read 0x1f27d8d thru 0x1f33761 (0xb9d5 bytes) 2024-09-16T12:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout:660 write 0x1d2e0b7 thru 0x1d2f0c0 (0x100a bytes) 2024-09-16T12:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout:661 write 0x6b69028 thru 0x6b6cd4d (0x3d26 bytes) 2024-09-16T12:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout:663 punch from 0xdda0ac to 0xddb205, (0x1159 bytes) 2024-09-16T12:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout:664 trunc from 0x77f687e to 0xe11c374 2024-09-16T12:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout:666 write 0x1dc5ffd thru 0x1dd2ce0 (0xcce4 bytes) 2024-09-16T12:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout:669 write 0xbe7a05e thru 0xbe8048f (0x6432 bytes) 2024-09-16T12:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout:670 trunc from 0xe11c374 to 0xc0f06d4 2024-09-16T12:59:28.221 INFO:teuthology.orchestra.run.smithi042.stdout:671 write 0x9ff0f7f thru 0x9ff445f (0x34e1 bytes) 2024-09-16T12:59:28.224 INFO:teuthology.orchestra.run.smithi042.stdout:672 writesame 0x76aa0c2 thru 0x76b763a data_size 0x1(0xd579 bytes) 2024-09-16T12:59:28.235 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:28.227+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:28.242 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:28.243 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:28.243 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:28.243 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:28.252 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:28.261 INFO:teuthology.orchestra.run.smithi042.stdout:673 trunc from 0xc0f06d4 to 0xc5f2aba 2024-09-16T12:59:28.273 INFO:teuthology.orchestra.run.smithi042.stdout:674 punch from 0x13a2fec to 0x13ac431, (0x9445 bytes) 2024-09-16T12:59:28.273 INFO:teuthology.orchestra.run.smithi042.stdout:677 read 0x98d3d37 thru 0x98d6fbc (0x3286 bytes) 2024-09-16T12:59:28.274 INFO:teuthology.orchestra.run.smithi042.stdout:678 writesame 0xafb1d09 thru 0xafb7b9f data_size 0x1d(0x5e97 bytes) 2024-09-16T12:59:28.280 INFO:teuthology.orchestra.run.smithi042.stdout:679 read 0x205761b thru 0x205998d (0x2373 bytes) 2024-09-16T12:59:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:680 write 0x5474703 thru 0x5474fb4 (0x8b2 bytes) 2024-09-16T12:59:28.284 INFO:teuthology.orchestra.run.smithi042.stdout:681 punch from 0xb55995f to 0xb563e86, (0xa527 bytes) 2024-09-16T12:59:28.284 INFO:teuthology.orchestra.run.smithi042.stdout:682 punch from 0x31c99af to 0x31d7058, (0xd6a9 bytes) 2024-09-16T12:59:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:686 trunc from 0xc5f2aba to 0x9172e2d 2024-09-16T12:59:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:687 read 0x72e8d96 thru 0x72f406f (0xb2da bytes) 2024-09-16T12:59:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:688 punch from 0x526e637 to 0x5276cf5, (0x86be bytes) 2024-09-16T12:59:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:691 trunc from 0x9172e2d to 0xa989719 2024-09-16T12:59:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:692 read 0xa878de4 thru 0xa87acdf (0x1efc bytes) 2024-09-16T12:59:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:693 punch from 0x5f98839 to 0x5fa69ba, (0xe181 bytes) 2024-09-16T12:59:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:694 read 0x3305341 thru 0x331458a (0xf24a bytes) 2024-09-16T12:59:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:696 read 0x1cbe74a thru 0x1cc70ed (0x89a4 bytes) 2024-09-16T12:59:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:697 trunc from 0xa989719 to 0x98799ba 2024-09-16T12:59:28.336 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:28.331+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:698 write 0x76035c4 thru 0x7609f92 (0x69cf bytes) 2024-09-16T12:59:28.353 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:28.353 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:28.353 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:28.353 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:28.353 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:699 write 0xab5ffe3 thru 0xab6bffa (0xc018 bytes) 2024-09-16T12:59:28.362 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:700 write 0x223e51a thru 0x224a5b1 (0xc098 bytes) 2024-09-16T12:59:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:703 write 0x8aa1c62 thru 0x8ab141a (0xf7b9 bytes) 2024-09-16T12:59:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:705 write 0xac1b85a thru 0xac2aafa (0xf2a1 bytes) 2024-09-16T12:59:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:706 write 0x111bb4c thru 0x1126440 (0xa8f5 bytes) 2024-09-16T12:59:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:707 read 0x434b2ae thru 0x434d11f (0x1e72 bytes) 2024-09-16T12:59:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:710 read 0x43e6ac4 thru 0x43f10d8 (0xa615 bytes) 2024-09-16T12:59:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:711 punch from 0x895f4d0 to 0x896362f, (0x415f bytes) 2024-09-16T12:59:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:712 read 0xa36b119 thru 0xa37260b (0x74f3 bytes) 2024-09-16T12:59:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:713 read 0xe6b73b thru 0xe78a13 (0xd2d9 bytes) 2024-09-16T12:59:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:715 trunc from 0xac2aafb to 0x6b55f17 2024-09-16T12:59:28.428 INFO:teuthology.orchestra.run.smithi042.stdout:716 read 0x57af4fd thru 0x57b7584 (0x8088 bytes) 2024-09-16T12:59:28.429 INFO:teuthology.orchestra.run.smithi042.stdout:717 read 0x1c66528 thru 0x1c6860e (0x20e7 bytes) 2024-09-16T12:59:28.430 INFO:teuthology.orchestra.run.smithi042.stdout:718 read 0x2d9692c thru 0x2d98085 (0x175a bytes) 2024-09-16T12:59:28.437 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:28.427+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:28.460 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.669871918, 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089374989999999998, 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 11 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785978+0000", 2024-09-16T12:59:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.373e-06 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785989+0000", 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1596999999999999e-05 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786008+0000", 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8510999999999999e-05 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786016+0000", 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786102+0000", 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6339000000000001e-05 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786113+0000", 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1386000000000001e-05 2024-09-16T12:59:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.788774+0000", 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026609519999999998 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.788888+0000", 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113189 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789043+0000", 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015570800000000001 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794705+0000", 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056619640000000002 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794770+0000", 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4789999999999995e-05 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794853+0000", 2024-09-16T12:59:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3441000000000003e-05 2024-09-16T12:59:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794913+0000", 2024-09-16T12:59:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9157999999999998e-05 2024-09-16T12:59:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:485 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.347664133, 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066273469999999996, 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 485 2024-09-16T12:59:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108185+0000", 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3700000000000002e-06 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108195+0000", 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2839999999999995e-06 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108220+0000", 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5066999999999999e-05 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108229+0000", 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5019999999999998e-06 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108334+0000", 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105136 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108571+0000", 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236495 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.109793+0000", 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012221809999999999 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114733+0000", 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049399350000000003 2024-09-16T12:59:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114745+0000", 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2228999999999999e-05 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114783+0000", 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7716999999999998e-05 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114810+0000", 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7430999999999999e-05 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:635 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.982261632, 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046232310000000002, 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 635 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473588+0000", 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9529999999999999e-06 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473602+0000", 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3266e-05 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473632+0000", 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0015e-05 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473641+0000", 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5359999999999998e-06 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473734+0000", 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3286000000000001e-05 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473875+0000", 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140211 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.474719+0000", 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084420100000000004 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.477891+0000", 2024-09-16T12:59:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031721079999999999 2024-09-16T12:59:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478108+0000", 2024-09-16T12:59:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021716899999999999 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478167+0000", 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8428999999999999e-05 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478209+0000", 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2057000000000001e-05 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1248 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'11, mlcod=39'8)", 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4278050479999997, 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053616649999999998, 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028044+0000", 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5e-06 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028058+0000", 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4469e-05 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028091+0000", 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2517000000000001e-05 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028106+0000", 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5116000000000001e-05 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028219+0000", 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113768 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028268+0000", 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8745000000000002e-05 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033313+0000", 2024-09-16T12:59:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050452869999999999 2024-09-16T12:59:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033404+0000", 2024-09-16T12:59:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0263e-05 2024-09-16T12:59:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1249 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'12, mlcod=39'8)", 2024-09-16T12:59:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.029057+0000", 2024-09-16T12:59:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4267899199999992, 2024-09-16T12:59:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044055769999999999, 2024-09-16T12:59:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029057+0000", 2024-09-16T12:59:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029057+0000", 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029059+0000", 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5370000000000001e-06 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029070+0000", 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1593000000000001e-05 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029095+0000", 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4505999999999999e-05 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029105+0000", 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7769999999999997e-06 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029214+0000", 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109887 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029273+0000", 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8709000000000003e-05 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033408+0000", 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041346880000000001 2024-09-16T12:59:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033463+0000", 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4880000000000003e-05 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1250 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'13, mlcod=39'8)", 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.425483968, 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061776180000000002, 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030364+0000", 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.356e-06 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030375+0000", 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0053e-05 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030395+0000", 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0767e-05 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030404+0000", 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4430000000000003e-06 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030492+0000", 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7842000000000004e-05 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030514+0000", 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2819999999999998e-05 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036453+0000", 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059381169999999997 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036541+0000", 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8220000000000006e-05 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1251 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'14, mlcod=39'8)", 2024-09-16T12:59:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.032254+0000", 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4235932160000004, 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043426029999999996, 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032254+0000", 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032254+0000", 2024-09-16T12:59:28.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032255+0000", 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1000000000000001e-06 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032263+0000", 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5660000000000001e-06 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032282+0000", 2024-09-16T12:59:28.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9184000000000001e-05 2024-09-16T12:59:28.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032290+0000", 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6340000000000007e-06 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032330+0000", 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9375000000000002e-05 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.032349+0000", 2024-09-16T12:59:28.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9612000000000001e-05 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036546+0000", 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041971630000000003 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036596+0000", 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9969000000000002e-05 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.502 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1252 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'15, mlcod=39'9)", 2024-09-16T12:59:28.502 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.033997+0000", 2024-09-16T12:59:28.502 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4218496989999991, 2024-09-16T12:59:28.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047268359999999999, 2024-09-16T12:59:28.502 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.502 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.502 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033997+0000", 2024-09-16T12:59:28.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033997+0000", 2024-09-16T12:59:28.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033998+0000", 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.054e-06 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034023+0000", 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5041e-05 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034048+0000", 2024-09-16T12:59:28.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4097999999999999e-05 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034058+0000", 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0515e-05 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034128+0000", 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0362000000000002e-05 2024-09-16T12:59:28.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034165+0000", 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6600999999999997e-05 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.038652+0000", 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044872999999999996 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.038724+0000", 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1865000000000004e-05 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1253 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'16, mlcod=39'11)", 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.035656+0000", 2024-09-16T12:59:28.507 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4201907509999998, 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038991009999999999, 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035656+0000", 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035656+0000", 2024-09-16T12:59:28.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035657+0000", 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0100000000000001e-06 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035665+0000", 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1499999999999999e-06 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035682+0000", 2024-09-16T12:59:28.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6351e-05 2024-09-16T12:59:28.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035689+0000", 2024-09-16T12:59:28.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0539999999999997e-06 2024-09-16T12:59:28.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035750+0000", 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1097000000000001e-05 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.035785+0000", 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4773000000000001e-05 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.039498+0000", 2024-09-16T12:59:28.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037130800000000001 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.039555+0000", 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7586000000000001e-05 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1263 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'19, mlcod=39'15)", 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.047178+0000", 2024-09-16T12:59:28.512 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4086691729999998, 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045177070000000001, 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047178+0000", 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047178+0000", 2024-09-16T12:59:28.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047179+0000", 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000003e-07 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047197+0000", 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8629000000000001e-05 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047211+0000", 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3699999999999999e-05 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047218+0000", 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1040000000000001e-06 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047273+0000", 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4483e-05 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047297+0000", 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.438e-05 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051620+0000", 2024-09-16T12:59:28.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043228920000000001 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051696+0000", 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5604000000000003e-05 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1296 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'29, mlcod=39'25)", 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:28.517 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3767104410000002, 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004900616, 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1800000000000005e-07 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079145+0000", 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3760000000000002e-06 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079159+0000", 2024-09-16T12:59:28.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4438e-05 2024-09-16T12:59:28.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079168+0000", 2024-09-16T12:59:28.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2299999999999997e-06 2024-09-16T12:59:28.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079253+0000", 2024-09-16T12:59:28.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4686999999999997e-05 2024-09-16T12:59:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079299+0000", 2024-09-16T12:59:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6072000000000002e-05 2024-09-16T12:59:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084001+0000", 2024-09-16T12:59:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047020100000000004 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084037+0000", 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5985000000000002e-05 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1301 4.5 4:a692e91d:::rbd_data.11b53140b009.00000000000000c4:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:28.522 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3754467740000003, 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060951929999999996, 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1301 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080402+0000", 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.75e-06 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080409+0000", 2024-09-16T12:59:28.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2219999999999996e-06 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080429+0000", 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9527000000000001e-05 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080438+0000", 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7819999999999999e-06 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080503+0000", 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5438999999999995e-05 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080693+0000", 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190218 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080909+0000", 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021529000000000001 2024-09-16T12:59:28.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084016+0000", 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031074319999999998 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086352+0000", 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023357690000000001 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086446+0000", 2024-09-16T12:59:28.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4158000000000002e-05 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086495+0000", 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9605999999999999e-05 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1297 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'30, mlcod=39'27)", 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.081247+0000", 2024-09-16T12:59:28.528 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3746001099999994, 2024-09-16T12:59:28.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050937860000000003, 2024-09-16T12:59:28.529 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.529 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.529 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081247+0000", 2024-09-16T12:59:28.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081247+0000", 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081248+0000", 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7199999999999997e-07 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081255+0000", 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2169999999999997e-06 2024-09-16T12:59:28.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081272+0000", 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6671e-05 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081278+0000", 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4350000000000002e-06 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081341+0000", 2024-09-16T12:59:28.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3001000000000002e-05 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081365+0000", 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3247e-05 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086290+0000", 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049256860000000003 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086341+0000", 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0556999999999997e-05 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1298 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'31, mlcod=39'27)", 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.083232+0000", 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3726154210000008, 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039085630000000003, 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.533 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083232+0000", 2024-09-16T12:59:28.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083232+0000", 2024-09-16T12:59:28.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083233+0000", 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0020000000000001e-06 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083243+0000", 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0441999999999999e-05 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083267+0000", 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3706999999999999e-05 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083276+0000", 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4239999999999999e-06 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083345+0000", 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8571999999999993e-05 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083367+0000", 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2405999999999999e-05 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.087081+0000", 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037133299999999999 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.087140+0000", 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9679999999999998e-05 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:3618 4.a 4:51b44bfb:::rbd_data.11b5c378985b.0000000000000003:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:22.210677+0000", 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.2451700280000004, 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045806279999999998, 2024-09-16T12:59:28.538 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3618 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210677+0000", 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210677+0000", 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210680+0000", 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8899999999999999e-06 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210699+0000", 2024-09-16T12:59:28.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9349999999999999e-05 2024-09-16T12:59:28.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210736+0000", 2024-09-16T12:59:28.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6631999999999998e-05 2024-09-16T12:59:28.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210750+0000", 2024-09-16T12:59:28.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3757000000000001e-05 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210924+0000", 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017455300000000001 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.211175+0000", 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025073700000000001 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.211578+0000", 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040305499999999998 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.213135+0000", 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001557206 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.215114+0000", 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019784210000000002 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.215200+0000", 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6254000000000005e-05 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.215258+0000", 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7772999999999997e-05 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.544 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:3722 4.a 4:57b538f8:::rbd_header.11b5d5152bdd:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:22.903924+0000", 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.5519230659999996, 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068689479999999997, 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3722 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903924+0000", 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903924+0000", 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903928+0000", 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5099999999999999e-06 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903944+0000", 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6422999999999998e-05 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903982+0000", 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8025999999999998e-05 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.904003+0000", 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0843e-05 2024-09-16T12:59:28.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.904124+0000", 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012166 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.904135+0000", 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0703999999999999e-05 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.908409+0000", 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042740410000000001 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.908554+0000", 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014520900000000001 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.908771+0000", 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021657999999999999 2024-09-16T12:59:28.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.909753+0000", 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0009819430000000001 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.910682+0000", 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092942399999999998 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.910744+0000", 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1796000000000003e-05 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.910793+0000", 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8789000000000003e-05 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:4074 4.a 4:57b538f8:::rbd_header.11b5d5152bdd:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:23.843397+0000", 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6124500790000003, 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078647490000000007, 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4074 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843397+0000", 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843397+0000", 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843400+0000", 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5560000000000002e-06 2024-09-16T12:59:28.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843409+0000", 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6120000000000005e-06 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843434+0000", 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5125000000000001e-05 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843448+0000", 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3550000000000001e-05 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843527+0000", 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9359999999999999e-05 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843534+0000", 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8220000000000001e-06 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.847907+0000", 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043724819999999996 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.848103+0000", 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019643899999999999 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.848776+0000", 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067343899999999996 2024-09-16T12:59:28.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.850197+0000", 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014204269999999999 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.850995+0000", 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00079860300000000005 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.851155+0000", 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159479 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.851262+0000", 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010685500000000001 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:28.558 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.091653357, 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:28.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T12:59:28.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5076 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.0039356859999999, 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020443647999999998, 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5076 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451913+0000", 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451920+0000", 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5459999999999997e-06 2024-09-16T12:59:28.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451938+0000", 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7581999999999999e-05 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451950+0000", 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2033e-05 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452042+0000", 2024-09-16T12:59:28.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2533000000000003e-05 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452251+0000", 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020887 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.456761+0000", 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045099959999999996 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472274+0000", 2024-09-16T12:59:28.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015512882 2024-09-16T12:59:28.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472281+0000", 2024-09-16T12:59:28.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3939999999999998e-06 2024-09-16T12:59:28.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472324+0000", 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2389999999999999e-05 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472355+0000", 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1436000000000002e-05 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5232 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [write 26019337~28881] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:28.569 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.900074+0000", 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.55577302200000001, 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058250639999999996, 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5232 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:28.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900074+0000", 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900074+0000", 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900077+0000", 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7700000000000002e-06 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900113+0000", 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6511999999999999e-05 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900156+0000", 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2483e-05 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900177+0000", 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1192999999999999e-05 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900300+0000", 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012320699999999999 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900310+0000", 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6099999999999995e-06 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.903745+0000", 2024-09-16T12:59:28.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034353640000000002 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.903815+0000", 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9387999999999994e-05 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.903903+0000", 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8127999999999994e-05 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.904766+0000", 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086280999999999999 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.905841+0000", 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00107515 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.905877+0000", 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6720000000000001e-05 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:28.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:28.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:28.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.905899+0000", 2024-09-16T12:59:28.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1729000000000002e-05 2024-09-16T12:59:28.576 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.576 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.576 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.576 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:28.576 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:28.576 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:28.578 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:28.579 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:28.571+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:28.579 INFO:teuthology.orchestra.run.smithi042.stdout:719 write 0xe2563b1 thru 0xe2643b1 (0xe001 bytes) 2024-09-16T12:59:28.579 INFO:teuthology.orchestra.run.smithi042.stdout:721 read 0x1b15c21 thru 0x1b2536d (0xf74d bytes) 2024-09-16T12:59:28.579 INFO:teuthology.orchestra.run.smithi042.stdout:723 read 0xa9ba1eb thru 0xa9c054b (0x6361 bytes) 2024-09-16T12:59:28.579 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.579 INFO:teuthology.orchestra.run.smithi042.stdout:727 writesame 0x6e1a672 thru 0x6e27291 data_size 0x2(0xcc20 bytes) 2024-09-16T12:59:28.580 INFO:teuthology.orchestra.run.smithi042.stdout:730 write 0xdfcba03 thru 0xdfd64ff (0xaafd bytes) 2024-09-16T12:59:28.580 INFO:teuthology.orchestra.run.smithi042.stdout:732 trunc from 0xe2643b2 to 0xc6f439 2024-09-16T12:59:28.580 INFO:teuthology.orchestra.run.smithi042.stdout:734 trunc from 0xc6f439 to 0x7b0df12 2024-09-16T12:59:28.587 INFO:teuthology.orchestra.run.smithi042.stdout:736 writesame 0x591f3e5 thru 0x592985f data_size 0x29(0xa47b bytes) 2024-09-16T12:59:28.594 INFO:teuthology.orchestra.run.smithi042.stdout:737 read 0x7993997 thru 0x799be38 (0x84a2 bytes) 2024-09-16T12:59:28.594 INFO:teuthology.orchestra.run.smithi042.stdout:739 read 0x2d79cb8 thru 0x2d7a37c (0x6c5 bytes) 2024-09-16T12:59:28.595 INFO:teuthology.orchestra.run.smithi042.stdout:741 writesame 0x13913b1 thru 0x1391f5a data_size 0x1(0xbaa bytes) 2024-09-16T12:59:28.604 INFO:teuthology.orchestra.run.smithi042.stdout:742 write 0x93d941 thru 0x94d7ba (0xfe7a bytes) 2024-09-16T12:59:28.607 INFO:teuthology.orchestra.run.smithi042.stdout:744 punch from 0x308a889 to 0x308cdd3, (0x254a bytes) 2024-09-16T12:59:28.607 INFO:teuthology.orchestra.run.smithi042.stdout:745 read 0x31aa30 thru 0x3222af (0x7880 bytes) 2024-09-16T12:59:28.609 INFO:teuthology.orchestra.run.smithi042.stdout:748 punch from 0x3b4c010 to 0x3b59ba2, (0xdb92 bytes) 2024-09-16T12:59:28.609 INFO:teuthology.orchestra.run.smithi042.stdout:749 write 0x397cbe7 thru 0x398aa46 (0xde60 bytes) 2024-09-16T12:59:28.613 INFO:teuthology.orchestra.run.smithi042.stdout:750 punch from 0x6085f1 to 0x6132a2, (0xacb1 bytes) 2024-09-16T12:59:28.613 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.613 INFO:teuthology.orchestra.run.smithi042.stdout:752 writesame 0xe1e56e thru 0xe26f25 data_size 0x71(0x89b8 bytes) 2024-09-16T12:59:28.616 INFO:teuthology.orchestra.run.smithi042.stdout:753 punch from 0x71ad4ed to 0x71b913a, (0xbc4d bytes) 2024-09-16T12:59:28.639 INFO:teuthology.orchestra.run.smithi042.stdout:754 write 0xb745875 thru 0xb752dcd (0xd559 bytes) 2024-09-16T12:59:28.641 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:28.646 INFO:teuthology.orchestra.run.smithi042.stdout:755 read 0xb0d2ddf thru 0xb0d5e5d (0x307f bytes) 2024-09-16T12:59:28.659 INFO:teuthology.orchestra.run.smithi042.stdout:756 write 0xb7afe69 thru 0xb7b9bdd (0x9d75 bytes) 2024-09-16T12:59:28.661 INFO:teuthology.orchestra.run.smithi042.stdout:757 punch from 0x60b34c0 to 0x60b893f, (0x547f bytes) 2024-09-16T12:59:28.678 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:28.675+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:28.681 INFO:teuthology.orchestra.run.smithi042.stdout:759 writesame 0xe51e1ae thru 0xe522b4c data_size 0x1(0x499f bytes) 2024-09-16T12:59:28.681 INFO:teuthology.orchestra.run.smithi042.stdout:760 write 0x8404fdd thru 0x8406584 (0x15a8 bytes) 2024-09-16T12:59:28.688 INFO:teuthology.orchestra.run.smithi042.stdout:761 read 0xfeadc7 thru 0xff6573 (0xb7ad bytes) 2024-09-16T12:59:28.690 INFO:teuthology.orchestra.run.smithi042.stdout:762 writesame 0xcb89975 thru 0xcb948fe data_size 0x1(0xaf8a bytes) 2024-09-16T12:59:28.728 INFO:teuthology.orchestra.run.smithi042.stdout:764 read 0x7cf7f38 thru 0x7cfbddf (0x3ea8 bytes) 2024-09-16T12:59:28.729 INFO:teuthology.orchestra.run.smithi042.stdout:766 punch from 0xa1c6d22 to 0xa1c93d5, (0x26b3 bytes) 2024-09-16T12:59:28.739 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:28.739 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:28.740 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:28.740 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:28.740 INFO:teuthology.orchestra.run.smithi042.stdout:767 trunc from 0xe522b4d to 0x8bd7f50 2024-09-16T12:59:28.750 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:28.764 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.765 INFO:teuthology.orchestra.run.smithi042.stdout:769 write 0x7b32392 thru 0x7b394bf (0x712e bytes) 2024-09-16T12:59:28.767 INFO:teuthology.orchestra.run.smithi042.stdout:770 punch from 0x255f2c to 0x263d01, (0xddd5 bytes) 2024-09-16T12:59:28.779 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:28.782 INFO:teuthology.orchestra.run.smithi042.stdout:771 write 0xc659717 thru 0xc664d06 (0xb5f0 bytes) 2024-09-16T12:59:28.788 INFO:teuthology.orchestra.run.smithi042.stdout:772 writesame 0x9de506f thru 0x9df2d1f data_size 0x7(0xdcb1 bytes) 2024-09-16T12:59:28.794 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:28.791+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:28.799 INFO:teuthology.orchestra.run.smithi042.stdout:773 write 0x3ae4e67 thru 0x3aeffc4 (0xb15e bytes) 2024-09-16T12:59:28.802 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:28.820 INFO:teuthology.orchestra.run.smithi042.stdout:774 trunc from 0xc664d07 to 0x1f5fa18 2024-09-16T12:59:28.831 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:28.847 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:28.847 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:28.853 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:28.856 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:28.863 INFO:teuthology.orchestra.run.smithi042.stdout:775 writesame 0x8586acf thru 0x8590a3b data_size 0x1(0x9f6d bytes) 2024-09-16T12:59:28.863 INFO:teuthology.orchestra.run.smithi042.stdout:776 punch from 0x2c3a12f to 0x2c47173, (0xd044 bytes) 2024-09-16T12:59:28.873 INFO:teuthology.orchestra.run.smithi042.stdout:777 trunc from 0x8590a3c to 0x29d4924 2024-09-16T12:59:28.874 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:28.880 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:28.871+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T12:59:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T12:59:28.894 INFO:teuthology.orchestra.run.smithi042.stdout:779 read 0x1eed560 thru 0x1ef27ea (0x528b bytes) 2024-09-16T12:59:28.901 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:28.904 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T12:59:28.912 INFO:teuthology.orchestra.run.smithi042.stdout:780 write 0x64646f1 thru 0x646872c (0x403c bytes) 2024-09-16T12:59:28.917 INFO:teuthology.orchestra.run.smithi042.stdout:781 read 0x39fefc0 thru 0x3a05513 (0x6554 bytes) 2024-09-16T12:59:28.918 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:28.918 INFO:teuthology.orchestra.run.smithi042.stdout:784 punch from 0x48fe8e5 to 0x4901311, (0x2a2c bytes) 2024-09-16T12:59:28.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:28.928 INFO:teuthology.orchestra.run.smithi042.stdout:785 trunc from 0x646872d to 0xc2a84bd 2024-09-16T12:59:28.937 INFO:teuthology.orchestra.run.smithi042.stdout:786 write 0x5e424e8 thru 0x5e4359b (0x10b4 bytes) 2024-09-16T12:59:28.942 INFO:teuthology.orchestra.run.smithi042.stdout:787 writesame 0x316d362 thru 0x3170d4f data_size 0x5(0x39ee bytes) 2024-09-16T12:59:28.944 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:28.946 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:28.946 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:28.946 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:28.946 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:28.946 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:28.947 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:28.947 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:28.947 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:28.947 INFO:teuthology.orchestra.run.smithi042.stdout:788 write 0x722e07b thru 0x7238cb7 (0xac3d bytes) 2024-09-16T12:59:28.949 INFO:teuthology.orchestra.run.smithi042.stdout:789 write 0x97efff4 thru 0x97f5f73 (0x5f80 bytes) 2024-09-16T12:59:28.964 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:28.964 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:28.973 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:28.974 INFO:teuthology.orchestra.run.smithi042.stdout:790 trunc from 0xc2a84bd to 0xe5019d 2024-09-16T12:59:28.998 INFO:teuthology.orchestra.run.smithi042.stdout:791 read 0x5be47 thru 0x627de (0x6998 bytes) 2024-09-16T12:59:28.998 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:28.991+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:28.999 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.016 INFO:teuthology.orchestra.run.smithi042.stdout:794 write 0x567bcc2 thru 0x568a1ec (0xe52b bytes) 2024-09-16T12:59:29.033 INFO:teuthology.orchestra.run.smithi042.stdout:795 write 0x5c44eba thru 0x5c4f7b8 (0xa8ff bytes) 2024-09-16T12:59:29.036 INFO:teuthology.orchestra.run.smithi042.stdout:797 read 0x4a1123d thru 0x4a1e10e (0xced2 bytes) 2024-09-16T12:59:29.036 INFO:teuthology.orchestra.run.smithi042.stdout:800 punch from 0x1d87c6b to 0x1d95f19, (0xe2ae bytes) 2024-09-16T12:59:29.036 INFO:teuthology.orchestra.run.smithi042.stdout:801 read 0x2ba4b2e thru 0x2bb3479 (0xe94c bytes) 2024-09-16T12:59:29.036 INFO:teuthology.orchestra.run.smithi042.stdout:802 punch from 0x4b3ffa4 to 0x4b4fb29, (0xfb85 bytes) 2024-09-16T12:59:29.042 INFO:teuthology.orchestra.run.smithi042.stdout:803 trunc from 0x5c4f7b9 to 0x26c9849 2024-09-16T12:59:29.064 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.064 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.081 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:29.075+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:29.082 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:29.082 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:29.090 INFO:teuthology.orchestra.run.smithi042.stdout:806 writesame 0xb8abcba thru 0xb8b19c1 data_size 0x34(0x5d08 bytes) 2024-09-16T12:59:29.090 INFO:teuthology.orchestra.run.smithi042.stdout:809 flatten 2024-09-16T12:59:29.093 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:29.117 INFO:teuthology.orchestra.run.smithi042.stdout:810 write 0xd87098c thru 0xd87f447 (0xeabc bytes) 2024-09-16T12:59:29.125 INFO:teuthology.orchestra.run.smithi042.stdout:812 writesame 0x26159f3 thru 0x2618be9 data_size 0x1(0x31f7 bytes) 2024-09-16T12:59:29.134 INFO:teuthology.orchestra.run.smithi042.stdout:813 read 0x6517084 thru 0x6519360 (0x22dd bytes) 2024-09-16T12:59:29.145 INFO:teuthology.orchestra.run.smithi042.stdout:815 trunc from 0xd87f448 to 0x8471fc0 2024-09-16T12:59:29.163 INFO:teuthology.orchestra.run.smithi042.stdout:816 read 0x759eff3 thru 0x75a4e92 (0x5ea0 bytes) 2024-09-16T12:59:29.163 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.163 INFO:teuthology.orchestra.run.smithi042.stdout:818 read 0x7923ea4 thru 0x7928db7 (0x4f14 bytes) 2024-09-16T12:59:29.183 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:29.175+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:29.184 INFO:teuthology.orchestra.run.smithi042.stdout:819 write 0xebfa2d8 thru 0xec04665 (0xa38e bytes) 2024-09-16T12:59:29.192 INFO:teuthology.orchestra.run.smithi042.stdout:820 read 0x398f4f4 thru 0x399f1e5 (0xfcf2 bytes) 2024-09-16T12:59:29.192 INFO:teuthology.orchestra.run.smithi042.stdout:821 read 0x3fb8e38 thru 0x3fbc66d (0x3836 bytes) 2024-09-16T12:59:29.192 INFO:teuthology.orchestra.run.smithi042.stdout:822 writesame 0xe92aef thru 0xe971d1 data_size 0x17(0x46e3 bytes) 2024-09-16T12:59:29.196 INFO:teuthology.orchestra.run.smithi042.stdout:824 write 0x1409858 thru 0x14158d0 (0xc079 bytes) 2024-09-16T12:59:29.203 INFO:teuthology.orchestra.run.smithi042.stdout:826 punch from 0xa6e45ec to 0xa6e8f40, (0x4954 bytes) 2024-09-16T12:59:29.206 INFO:teuthology.orchestra.run.smithi042.stdout:827 trunc from 0xec04666 to 0xd6c98de 2024-09-16T12:59:29.233 INFO:teuthology.orchestra.run.smithi042.stdout:828 trunc from 0xd6c98de to 0x507c00e 2024-09-16T12:59:29.236 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:29.236 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:29.246 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:29.254 INFO:teuthology.orchestra.run.smithi042.stdout:829 read 0x14c40fb thru 0x14ccbba (0x8ac0 bytes) 2024-09-16T12:59:29.269 INFO:teuthology.orchestra.run.smithi042.stdout:830 write 0x682ee9b thru 0x6834b05 (0x5c6b bytes) 2024-09-16T12:59:29.277 INFO:teuthology.orchestra.run.smithi042.stdout:831 write 0x3ae046d thru 0x3ae22db (0x1e6f bytes) 2024-09-16T12:59:29.283 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:29.279+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:29.306 INFO:teuthology.orchestra.run.smithi042.stdout:832 write 0xe4ebf80 thru 0xe4f73c9 (0xb44a bytes) 2024-09-16T12:59:29.314 INFO:teuthology.orchestra.run.smithi042.stdout:833 punch from 0xd374e3c to 0xd37ba3a, (0x6bfe bytes) 2024-09-16T12:59:29.314 INFO:teuthology.orchestra.run.smithi042.stdout:834 punch from 0xa342c02 to 0xa34bab1, (0x8eaf bytes) 2024-09-16T12:59:29.314 INFO:teuthology.orchestra.run.smithi042.stdout:836 write 0x8827c10 thru 0x88296cd (0x1abe bytes) 2024-09-16T12:59:29.320 INFO:teuthology.orchestra.run.smithi042.stdout:837 read 0x7af624f thru 0x7b02f9e (0xcd50 bytes) 2024-09-16T12:59:29.320 INFO:teuthology.orchestra.run.smithi042.stdout:838 punch from 0x21445f1 to 0x214e6d0, (0xa0df bytes) 2024-09-16T12:59:29.327 INFO:teuthology.orchestra.run.smithi042.stdout:839 trunc from 0xe4f73ca to 0xb09c357 2024-09-16T12:59:29.359 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:29.360 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:29.365 INFO:teuthology.orchestra.run.smithi042.stdout:840 trunc from 0xb09c357 to 0x7e28bb 2024-09-16T12:59:29.372 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:29.415 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:29.407+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:29.415 INFO:teuthology.orchestra.run.smithi042.stdout:841 writesame 0x1ced5f7 thru 0x1cf1876 data_size 0x70(0x4280 bytes) 2024-09-16T12:59:29.415 INFO:teuthology.orchestra.run.smithi042.stdout:843 punch from 0x12a5121 to 0x12b09ed, (0xb8cc bytes) 2024-09-16T12:59:29.415 INFO:teuthology.orchestra.run.smithi042.stdout:844 punch from 0xdfcb91 to 0xe05348, (0x87b7 bytes) 2024-09-16T12:59:29.415 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.416 INFO:teuthology.orchestra.run.smithi042.stdout:846 read 0x1966896 thru 0x196a3c3 (0x3b2e bytes) 2024-09-16T12:59:29.438 INFO:teuthology.orchestra.run.smithi042.stdout:847 trunc from 0x1cf1877 to 0xe7ef4c2 2024-09-16T12:59:29.449 INFO:teuthology.orchestra.run.smithi042.stdout:848 write 0x9a113c9 thru 0x9a19720 (0x8358 bytes) 2024-09-16T12:59:29.457 INFO:teuthology.orchestra.run.smithi042.stdout:849 punch from 0x13e511a to 0x13e545a, (0x340 bytes) 2024-09-16T12:59:29.457 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.457 INFO:teuthology.orchestra.run.smithi042.stdout:851 write 0xa7b1d17 thru 0xa7b5db7 (0x40a1 bytes) 2024-09-16T12:59:29.464 INFO:teuthology.orchestra.run.smithi042.stdout:853 read 0x1e3550b thru 0x1e3defc (0x89f2 bytes) 2024-09-16T12:59:29.465 INFO:teuthology.orchestra.run.smithi042.stdout:854 punch from 0x4e5c4c5 to 0x4e6a922, (0xe45d bytes) 2024-09-16T12:59:29.465 INFO:teuthology.orchestra.run.smithi042.stdout:855 read 0x60d737d thru 0x60decdd (0x7961 bytes) 2024-09-16T12:59:29.465 INFO:teuthology.orchestra.run.smithi042.stdout:856 read 0x9052be1 thru 0x905d624 (0xaa44 bytes) 2024-09-16T12:59:29.465 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.465 INFO:teuthology.orchestra.run.smithi042.stdout:858 write 0xd16cd40 thru 0xd172f8d (0x624e bytes) 2024-09-16T12:59:29.471 INFO:teuthology.orchestra.run.smithi042.stdout:859 trunc from 0xe7ef4c2 to 0xe273562 2024-09-16T12:59:29.483 INFO:teuthology.orchestra.run.smithi042.stdout:860 write 0xb6c99d8 thru 0xb6d44a3 (0xaacc bytes) 2024-09-16T12:59:29.485 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:29.479+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:29.486 INFO:teuthology.orchestra.run.smithi042.stdout:862 punch from 0x9a047ad to 0x9a0aafa, (0x634d bytes) 2024-09-16T12:59:29.487 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.487 INFO:teuthology.orchestra.run.smithi042.stdout:864 read 0x4754b25 thru 0x475bb10 (0x6fec bytes) 2024-09-16T12:59:29.487 INFO:teuthology.orchestra.run.smithi042.stdout:865 writesame 0xc79cfc1 thru 0xc7a1dc8 data_size 0x2c(0x4e08 bytes) 2024-09-16T12:59:29.494 INFO:teuthology.orchestra.run.smithi042.stdout:866 punch from 0xa75b938 to 0xa75c040, (0x708 bytes) 2024-09-16T12:59:29.494 INFO:teuthology.orchestra.run.smithi042.stdout:867 read 0x8394a63 thru 0x83a3ce2 (0xf280 bytes) 2024-09-16T12:59:29.495 INFO:teuthology.orchestra.run.smithi042.stdout:869 read 0xd4aaa93 thru 0xd4ad97b (0x2ee9 bytes) 2024-09-16T12:59:29.497 INFO:teuthology.orchestra.run.smithi042.stdout:871 punch from 0x769d9ef to 0x76a68ab, (0x8ebc bytes) 2024-09-16T12:59:29.497 INFO:teuthology.orchestra.run.smithi042.stdout:873 write 0x74c71ad thru 0x74ca2ed (0x3141 bytes) 2024-09-16T12:59:29.504 INFO:teuthology.orchestra.run.smithi042.stdout:877 write 0x92cd457 thru 0x92d435d (0x6f07 bytes) 2024-09-16T12:59:29.507 INFO:teuthology.orchestra.run.smithi042.stdout:878 read 0xadc7c76 thru 0xadd76b3 (0xfa3e bytes) 2024-09-16T12:59:29.508 INFO:teuthology.orchestra.run.smithi042.stdout:879 write 0x9570590 thru 0x9574494 (0x3f05 bytes) 2024-09-16T12:59:29.527 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:29.527 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:29.533 INFO:teuthology.orchestra.run.smithi042.stdout:880 trunc from 0xe273562 to 0x2344883 2024-09-16T12:59:29.539 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:29.557 INFO:teuthology.orchestra.run.smithi042.stdout:881 punch from 0x1bacc73 to 0x1bb76d0, (0xaa5d bytes) 2024-09-16T12:59:29.557 INFO:teuthology.orchestra.run.smithi042.stdout:882 write 0x9341f5 thru 0x93b2ec (0x70f8 bytes) 2024-09-16T12:59:29.584 INFO:teuthology.orchestra.run.smithi042.stdout:883 write 0xa2380d2 thru 0xa23b2f6 (0x3225 bytes) 2024-09-16T12:59:29.586 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:29.579+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:29.590 INFO:teuthology.orchestra.run.smithi042.stdout:884 punch from 0x78baf50 to 0x78bedad, (0x3e5d bytes) 2024-09-16T12:59:29.590 INFO:teuthology.orchestra.run.smithi042.stdout:886 read 0x7519234 thru 0x75217f9 (0x85c6 bytes) 2024-09-16T12:59:29.590 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.591 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.591 INFO:teuthology.orchestra.run.smithi042.stdout:891 punch from 0x89102a3 to 0x891f5a8, (0xf305 bytes) 2024-09-16T12:59:29.591 INFO:teuthology.orchestra.run.smithi042.stdout:893 read 0x342b8f7 thru 0x342d944 (0x204e bytes) 2024-09-16T12:59:29.591 INFO:teuthology.orchestra.run.smithi042.stdout:894 punch from 0x860a5b0 to 0x860e180, (0x3bd0 bytes) 2024-09-16T12:59:29.596 INFO:teuthology.orchestra.run.smithi042.stdout:896 trunc from 0xa23b2f7 to 0x78e62ec 2024-09-16T12:59:29.618 INFO:teuthology.orchestra.run.smithi042.stdout:897 read 0x7498c31 thru 0x74a1e9d (0x926d bytes) 2024-09-16T12:59:29.618 INFO:teuthology.orchestra.run.smithi042.stdout:899 read 0x343be2a thru 0x343caa3 (0xc7a bytes) 2024-09-16T12:59:29.627 INFO:teuthology.orchestra.run.smithi042.stdout:900 trunc from 0x78e62ec to 0xce6ae2a 2024-09-16T12:59:29.638 INFO:teuthology.orchestra.run.smithi042.stdout:901 read 0x60b95f2 thru 0x60c8eed (0xf8fc bytes) 2024-09-16T12:59:29.638 INFO:teuthology.orchestra.run.smithi042.stdout:902 write 0x94b3b12 thru 0x94b71f9 (0x36e8 bytes) 2024-09-16T12:59:29.646 INFO:teuthology.orchestra.run.smithi042.stdout:903 punch from 0xcba7381 to 0xcba9473, (0x20f2 bytes) 2024-09-16T12:59:29.646 INFO:teuthology.orchestra.run.smithi042.stdout:904 read 0x79d6ebd thru 0x79de2b8 (0x73fc bytes) 2024-09-16T12:59:29.646 INFO:teuthology.orchestra.run.smithi042.stdout:906 write 0x5de5f21 thru 0x5debbf0 (0x5cd0 bytes) 2024-09-16T12:59:29.646 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:29.647 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:29.654 INFO:teuthology.orchestra.run.smithi042.stdout:908 read 0x330cbe6 thru 0x331675e (0x9b79 bytes) 2024-09-16T12:59:29.654 INFO:teuthology.orchestra.run.smithi042.stdout:910 read 0xac5d762 thru 0xac6b644 (0xdee3 bytes) 2024-09-16T12:59:29.654 INFO:teuthology.orchestra.run.smithi042.stdout:911 write 0x48206a thru 0x485d38 (0x3ccf bytes) 2024-09-16T12:59:29.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:29.658 INFO:teuthology.orchestra.run.smithi042.stdout:913 read 0xaaebc76 thru 0xaaf1e58 (0x61e3 bytes) 2024-09-16T12:59:29.659 INFO:teuthology.orchestra.run.smithi042.stdout:914 writesame 0x94ca3a6 thru 0x94cef53 data_size 0x3(0x4bae bytes) 2024-09-16T12:59:29.665 INFO:teuthology.orchestra.run.smithi042.stdout:916 punch from 0x976099d to 0x9767b57, (0x71ba bytes) 2024-09-16T12:59:29.665 INFO:teuthology.orchestra.run.smithi042.stdout:917 write 0x270215e thru 0x270a510 (0x83b3 bytes) 2024-09-16T12:59:29.669 INFO:teuthology.orchestra.run.smithi042.stdout:918 writesame 0x14ab4b2 thru 0x14b89b2 data_size 0x1(0xd501 bytes) 2024-09-16T12:59:29.687 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:29.679+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:29.705 INFO:teuthology.orchestra.run.smithi042.stdout:919 writesame 0x1172d46 thru 0x117c629 data_size 0x4c(0x98e4 bytes) 2024-09-16T12:59:29.708 INFO:teuthology.orchestra.run.smithi042.stdout:920 read 0x3f5dd9c thru 0x3f6d4d6 (0xf73b bytes) 2024-09-16T12:59:29.708 INFO:teuthology.orchestra.run.smithi042.stdout:921 write 0x94c7b98 thru 0x94c829c (0x705 bytes) 2024-09-16T12:59:29.710 INFO:teuthology.orchestra.run.smithi042.stdout:922 read 0xbc18e31 thru 0xbc1ea1e (0x5bee bytes) 2024-09-16T12:59:29.711 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.711 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.711 INFO:teuthology.orchestra.run.smithi042.stdout:926 write 0xb3b83b6 thru 0xb3bc9da (0x4625 bytes) 2024-09-16T12:59:29.713 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.713 INFO:teuthology.orchestra.run.smithi042.stdout:928 read 0x61d4d13 thru 0x61db313 (0x6601 bytes) 2024-09-16T12:59:29.735 INFO:teuthology.orchestra.run.smithi042.stdout:929 trunc from 0xce6ae2a to 0x9eb83 2024-09-16T12:59:29.761 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:29.761 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:29.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:29.786 INFO:teuthology.orchestra.run.smithi042.stdout:930 write 0x653b5d1 thru 0x653cbdb (0x160b bytes) 2024-09-16T12:59:29.788 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:29.779+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:29.793 INFO:teuthology.orchestra.run.smithi042.stdout:931 read 0x2cbad61 thru 0x2cbf88c (0x4b2c bytes) 2024-09-16T12:59:29.794 INFO:teuthology.orchestra.run.smithi042.stdout:932 read 0x49fa1f1 thru 0x4a00b19 (0x6929 bytes) 2024-09-16T12:59:29.821 INFO:teuthology.orchestra.run.smithi042.stdout:933 writesame 0xc116007 thru 0xc121003 data_size 0x1(0xaffd bytes) 2024-09-16T12:59:29.836 INFO:teuthology.orchestra.run.smithi042.stdout:936 write 0xc9daed3 thru 0xc9ea429 (0xf557 bytes) 2024-09-16T12:59:29.844 INFO:teuthology.orchestra.run.smithi042.stdout:937 read 0x19bf5d9 thru 0x19c5e2f (0x6857 bytes) 2024-09-16T12:59:29.844 INFO:teuthology.orchestra.run.smithi042.stdout:938 write 0x940e9b7 thru 0x941822b (0x9875 bytes) 2024-09-16T12:59:29.851 INFO:teuthology.orchestra.run.smithi042.stdout:940 write 0xaa8b186 thru 0xaa8bce7 (0xb62 bytes) 2024-09-16T12:59:29.857 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:942 write 0xebbd9bf thru 0xebc5162 (0x77a4 bytes) 2024-09-16T12:59:29.876 INFO:teuthology.orchestra.run.smithi042.stdout:944 write 0x6386c56 thru 0x6387b07 (0xeb2 bytes) 2024-09-16T12:59:29.879 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.879 INFO:teuthology.orchestra.run.smithi042.stdout:946 read 0xae8d8fa thru 0xae92999 (0x50a0 bytes) 2024-09-16T12:59:29.890 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:29.879+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:29.891 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:29.891 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T12:59:29.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:29.907 INFO:teuthology.orchestra.run.smithi042.stdout:947 trunc from 0xebc5163 to 0x6795d5 2024-09-16T12:59:29.913 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T12:59:29.948 INFO:teuthology.orchestra.run.smithi042.stdout:948 write 0x3112f0d thru 0x3121dc0 (0xeeb4 bytes) 2024-09-16T12:59:29.955 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.955 INFO:teuthology.orchestra.run.smithi042.stdout:953 read 0x2d35810 thru 0x2d3d2d0 (0x7ac1 bytes) 2024-09-16T12:59:29.955 INFO:teuthology.orchestra.run.smithi042.stdout:955 read 0x2840f46 thru 0x284e1c1 (0xd27c bytes) 2024-09-16T12:59:29.970 INFO:teuthology.orchestra.run.smithi042.stdout:957 trunc from 0x3121dc1 to 0x9c7c649 2024-09-16T12:59:29.980 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:29.980 INFO:teuthology.orchestra.run.smithi042.stdout:959 read 0x95af76c thru 0x95b0b8f (0x1424 bytes) 2024-09-16T12:59:29.989 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:29.983+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:29.995 INFO:teuthology.orchestra.run.smithi042.stdout:961 write 0xb4a78ef thru 0xb4ae17c (0x688e bytes) 2024-09-16T12:59:30.017 INFO:teuthology.orchestra.run.smithi042.stdout:963 writesame 0xd8a0d33 thru 0xd8ae509 data_size 0x2b(0xd7d7 bytes) 2024-09-16T12:59:30.041 INFO:teuthology.orchestra.run.smithi042.stdout:964 trunc from 0xd8ae50a to 0x13f8d34 2024-09-16T12:59:30.060 INFO:teuthology.orchestra.run.smithi042.stdout:965 read 0x6a82f6 thru 0x6b166a (0x9375 bytes) 2024-09-16T12:59:30.061 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:30.061 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:30.073 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:30.076 INFO:teuthology.orchestra.run.smithi042.stdout:966 writesame 0x470d20d thru 0x471cdb2 data_size 0x1b(0xfba6 bytes) 2024-09-16T12:59:30.076 INFO:teuthology.orchestra.run.smithi042.stdout:969 writesame 0x869e60 thru 0x86c1d5 data_size 0x33(0x2376 bytes) 2024-09-16T12:59:30.089 INFO:teuthology.orchestra.run.smithi042.stdout:973 writesame 0x4a76296 thru 0x4a781d5 data_size 0x19(0x1f40 bytes) 2024-09-16T12:59:30.089 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:30.079+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:30.105 INFO:teuthology.orchestra.run.smithi042.stdout:974 trunc from 0x4a781d6 to 0xd774305 2024-09-16T12:59:30.115 INFO:teuthology.orchestra.run.smithi042.stdout:975 write 0x40f063f thru 0x40f524a (0x4c0c bytes) 2024-09-16T12:59:30.122 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.138 INFO:teuthology.orchestra.run.smithi042.stdout:978 trunc from 0xd774305 to 0x4acd53c 2024-09-16T12:59:30.153 INFO:teuthology.orchestra.run.smithi042.stdout:981 trunc from 0x4acd53c to 0x3278181 2024-09-16T12:59:30.177 INFO:teuthology.orchestra.run.smithi042.stdout:982 punch from 0x2e747ec to 0x2e7cdb7, (0x85cb bytes) 2024-09-16T12:59:30.185 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:30.185 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:30.191 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:30.187+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:30.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:30.205 INFO:teuthology.orchestra.run.smithi042.stdout:983 writesame 0xc8fb70b thru 0xc902c75 data_size 0x1(0x756b bytes) 2024-09-16T12:59:30.206 INFO:teuthology.orchestra.run.smithi042.stdout:984 write 0x8c69e13 thru 0x8c79d90 (0xff7e bytes) 2024-09-16T12:59:30.213 INFO:teuthology.orchestra.run.smithi042.stdout:988 read 0x77823eb thru 0x77840f0 (0x1d06 bytes) 2024-09-16T12:59:30.223 INFO:teuthology.orchestra.run.smithi042.stdout:989 writesame 0xcbb2c73 thru 0xcbb771d data_size 0x1(0x4aab bytes) 2024-09-16T12:59:30.223 INFO:teuthology.orchestra.run.smithi042.stdout:992 write 0x6c2ccb7 thru 0x6c34c8c (0x7fd6 bytes) 2024-09-16T12:59:30.249 INFO:teuthology.orchestra.run.smithi042.stdout:993 trunc from 0xcbb771e to 0x1aebddb 2024-09-16T12:59:30.270 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.292 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:30.283+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:30.302 INFO:teuthology.orchestra.run.smithi042.stdout:995 writesame 0xe7fde30 thru 0xe805c62 data_size 0xb(0x7e33 bytes) 2024-09-16T12:59:30.311 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:30.311 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:30.315 INFO:teuthology.orchestra.run.smithi042.stdout:996 trunc from 0xe805c63 to 0x5e97e2b 2024-09-16T12:59:30.323 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:30.328 INFO:teuthology.orchestra.run.smithi042.stdout:997 read 0x4e0399 thru 0x4e3ec2 (0x3b2a bytes) 2024-09-16T12:59:30.338 INFO:teuthology.orchestra.run.smithi042.stdout:998 trunc from 0x5e97e2b to 0x234f8d3 2024-09-16T12:59:30.354 INFO:teuthology.orchestra.run.smithi042.stdout:1000 read 0x21c9fdc thru 0x21d444f (0xa474 bytes) 2024-09-16T12:59:30.363 INFO:teuthology.orchestra.run.smithi042.stdout:1001 trunc from 0x234f8d3 to 0x6ebea07 2024-09-16T12:59:30.387 INFO:teuthology.orchestra.run.smithi042.stdout:1002 write 0x73cd438 thru 0x73d50fe (0x7cc7 bytes) 2024-09-16T12:59:30.392 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:30.387+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:30.417 INFO:teuthology.orchestra.run.smithi042.stdout:1003 write 0xc499582 thru 0xc499eb7 (0x936 bytes) 2024-09-16T12:59:30.430 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:30.430 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:30.438 INFO:teuthology.orchestra.run.smithi042.stdout:1004 trunc from 0xc499eb8 to 0x3a264c5 2024-09-16T12:59:30.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:30.485 INFO:teuthology.orchestra.run.smithi042.stdout:1005 writesame 0x884daf4 thru 0x885b353 data_size 0x3(0xd860 bytes) 2024-09-16T12:59:30.485 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.488 INFO:teuthology.orchestra.run.smithi042.stdout:1007 trunc from 0x885b354 to 0x74cb9dd 2024-09-16T12:59:30.493 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:30.487+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:30.500 INFO:teuthology.orchestra.run.smithi042.stdout:1008 write 0x11e610c thru 0x11f5570 (0xf465 bytes) 2024-09-16T12:59:30.504 INFO:teuthology.orchestra.run.smithi042.stdout:1009 read 0x57ae73 thru 0x585d33 (0xaec1 bytes) 2024-09-16T12:59:30.505 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.520 INFO:teuthology.orchestra.run.smithi042.stdout:1014 trunc from 0x74cb9dd to 0xd3b088 2024-09-16T12:59:30.556 INFO:teuthology.orchestra.run.smithi042.stdout:1015 trunc from 0xd3b088 to 0x8ec6955 2024-09-16T12:59:30.567 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.568 INFO:teuthology.orchestra.run.smithi042.stdout:1017 punch from 0x42f9af to 0x435195, (0x57e6 bytes) 2024-09-16T12:59:30.568 INFO:teuthology.orchestra.run.smithi042.stdout:1018 write 0x5891e70 thru 0x589ce19 (0xafaa bytes) 2024-09-16T12:59:30.579 INFO:teuthology.orchestra.run.smithi042.stdout:1019 trunc from 0x8ec6955 to 0x5c3cdb5 2024-09-16T12:59:30.594 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:30.594 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:30.594 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:30.591+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:30.597 INFO:teuthology.orchestra.run.smithi042.stdout:1021 trunc from 0x5c3cdb5 to 0x5328a90 2024-09-16T12:59:30.605 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:30.639 INFO:teuthology.orchestra.run.smithi042.stdout:1022 write 0x9ca0024 thru 0x9ca9dba (0x9d97 bytes) 2024-09-16T12:59:30.655 INFO:teuthology.orchestra.run.smithi042.stdout:1023 trunc from 0x9ca9dbb to 0x57a577a 2024-09-16T12:59:30.682 INFO:teuthology.orchestra.run.smithi042.stdout:1024 read 0xbbdbef thru 0xbc99b7 (0xbdc9 bytes) 2024-09-16T12:59:30.683 INFO:teuthology.orchestra.run.smithi042.stdout:1025 punch from 0x518168c to 0x51852a5, (0x3c19 bytes) 2024-09-16T12:59:30.695 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:30.687+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:30.718 INFO:teuthology.orchestra.run.smithi042.stdout:1026 write 0xe5aee71 thru 0xe5bc3cd (0xd55d bytes) 2024-09-16T12:59:30.722 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:30.723 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:30.725 INFO:teuthology.orchestra.run.smithi042.stdout:1027 read 0x6c8361c thru 0x6c8ac89 (0x766e bytes) 2024-09-16T12:59:30.733 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:30.737 INFO:teuthology.orchestra.run.smithi042.stdout:1028 write 0xe7421d3 thru 0xe749aab (0x78d9 bytes) 2024-09-16T12:59:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:1032 read 0xbcb872e thru 0xbcc340b (0xacde bytes) 2024-09-16T12:59:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:1034 write 0xa864448 thru 0xa868003 (0x3bbc bytes) 2024-09-16T12:59:30.746 INFO:teuthology.orchestra.run.smithi042.stdout:1037 write 0x951c0c8 thru 0x9521605 (0x553e bytes) 2024-09-16T12:59:30.752 INFO:teuthology.orchestra.run.smithi042.stdout:1039 write 0x2cce50f thru 0x2cd146c (0x2f5e bytes) 2024-09-16T12:59:30.753 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.753 INFO:teuthology.orchestra.run.smithi042.stdout:1042 read 0x1f94d4f thru 0x1f9d528 (0x87da bytes) 2024-09-16T12:59:30.754 INFO:teuthology.orchestra.run.smithi042.stdout:1044 writesame 0x43d7f6 thru 0x448343 data_size 0x3(0xab4e bytes) 2024-09-16T12:59:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:1046 punch from 0x1a8b9ce to 0x1a91be1, (0x6213 bytes) 2024-09-16T12:59:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:1048 write 0x68671de thru 0x6873ee8 (0xcd0b bytes) 2024-09-16T12:59:30.772 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.772 INFO:teuthology.orchestra.run.smithi042.stdout:1052 read 0x6f80c0e thru 0x6f81a28 (0xe1b bytes) 2024-09-16T12:59:30.772 INFO:teuthology.orchestra.run.smithi042.stdout:1053 write 0xe18564f thru 0xe186fc0 (0x1972 bytes) 2024-09-16T12:59:30.796 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:30.791+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:30.803 INFO:teuthology.orchestra.run.smithi042.stdout:1054 trunc from 0xe749aac to 0x158bc8 2024-09-16T12:59:30.837 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:30.837 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:30.839 INFO:teuthology.orchestra.run.smithi042.stdout:1055 writesame 0x1d323fc thru 0x1d40f28 data_size 0x1(0xeb2d bytes) 2024-09-16T12:59:30.840 INFO:teuthology.orchestra.run.smithi042.stdout:1056 trunc from 0x1d40f29 to 0x2313c49 2024-09-16T12:59:30.847 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:30.863 INFO:teuthology.orchestra.run.smithi042.stdout:1058 write 0x3994c9e thru 0x399cd99 (0x80fc bytes) 2024-09-16T12:59:30.888 INFO:teuthology.orchestra.run.smithi042.stdout:1059 trunc from 0x399cd9a to 0xb6e368c 2024-09-16T12:59:30.896 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:30.887+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:30.907 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T12:59:30.907 INFO:teuthology.orchestra.run.smithi042.stdout:1060 read 0x27cd644 thru 0x27d18cc (0x4289 bytes) 2024-09-16T12:59:30.908 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T12:59:30.908 INFO:teuthology.orchestra.run.smithi042.stdout:1061 read 0x925ccc thru 0x932bd6 (0xcf0b bytes) 2024-09-16T12:59:30.908 INFO:teuthology.orchestra.run.smithi042.stdout:1064 read 0x4727940 thru 0x4727ad3 (0x194 bytes) 2024-09-16T12:59:30.908 INFO:teuthology.orchestra.run.smithi042.stdout:1065 read 0x8f876bd thru 0x8f87fd4 (0x918 bytes) 2024-09-16T12:59:30.921 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T12:59:30.923 INFO:teuthology.orchestra.run.smithi042.stdout:1066 writesame 0xe0bb8ac thru 0xe0bd0af data_size 0x1d(0x1804 bytes) 2024-09-16T12:59:30.923 INFO:teuthology.orchestra.run.smithi042.stdout:1067 read 0x4d5d625 thru 0x4d665ce (0x8faa bytes) 2024-09-16T12:59:30.924 INFO:teuthology.orchestra.run.smithi042.stdout:1068 writesame 0x8e4a95d thru 0x8e4b236 data_size 0xb(0x8da bytes) 2024-09-16T12:59:30.929 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.929 INFO:teuthology.orchestra.run.smithi042.stdout:1070 write 0xa38735b thru 0xa38ee44 (0x7aea bytes) 2024-09-16T12:59:30.931 INFO:teuthology.orchestra.run.smithi042.stdout:1071 read 0x204d929 thru 0x204eef5 (0x15cd bytes) 2024-09-16T12:59:30.931 INFO:teuthology.orchestra.run.smithi042.stdout:1072 punch from 0x9637d33 to 0x963aa1c, (0x2ce9 bytes) 2024-09-16T12:59:30.931 INFO:teuthology.orchestra.run.smithi042.stdout:1073 write 0x36c5502 thru 0x36ccec3 (0x79c2 bytes) 2024-09-16T12:59:30.936 INFO:teuthology.orchestra.run.smithi042.stdout:1075 writesame 0x6cfe02f thru 0x6cfea63 data_size 0x3(0xa35 bytes) 2024-09-16T12:59:30.942 INFO:teuthology.orchestra.run.smithi042.stdout:1078 read 0x2f20040 thru 0x2f24658 (0x4619 bytes) 2024-09-16T12:59:30.942 INFO:teuthology.orchestra.run.smithi042.stdout:1081 write 0x32fbb00 thru 0x32fbc10 (0x111 bytes) 2024-09-16T12:59:30.945 INFO:teuthology.orchestra.run.smithi042.stdout:1083 write 0xf97670 thru 0xfa2b32 (0xb4c3 bytes) 2024-09-16T12:59:30.948 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:30.948 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:30.957 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:30.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:30.963 INFO:teuthology.orchestra.run.smithi042.stdout:1084 trunc from 0xe0bd0b0 to 0x4df6397 2024-09-16T12:59:30.982 INFO:teuthology.orchestra.run.smithi042.stdout:1085 punch from 0x3bc98fb to 0x3bcb8f0, (0x1ff5 bytes) 2024-09-16T12:59:30.982 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:30.982 INFO:teuthology.orchestra.run.smithi042.stdout:1087 punch from 0x207b8ec to 0x208357f, (0x7c93 bytes) 2024-09-16T12:59:30.982 INFO:teuthology.orchestra.run.smithi042.stdout:1088 read 0x3a95191 thru 0x3aa3d94 (0xec04 bytes) 2024-09-16T12:59:30.983 INFO:teuthology.orchestra.run.smithi042.stdout:1091 read 0x1aed3ac thru 0x1af7d59 (0xa9ae bytes) 2024-09-16T12:59:30.997 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:30.987+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:31.002 INFO:teuthology.orchestra.run.smithi042.stdout:1093 writesame 0xb031f6d thru 0xb03d632 data_size 0x5(0xb6c6 bytes) 2024-09-16T12:59:31.016 INFO:teuthology.orchestra.run.smithi042.stdout:1094 write 0xe2532d8 thru 0xe25c499 (0x91c2 bytes) 2024-09-16T12:59:31.022 INFO:teuthology.orchestra.run.smithi042.stdout:1095 punch from 0x2e36fe2 to 0x2e37438, (0x456 bytes) 2024-09-16T12:59:31.023 INFO:teuthology.orchestra.run.smithi042.stdout:1096 write 0xaa22228 thru 0xaa2d4ad (0xb286 bytes) 2024-09-16T12:59:31.029 INFO:teuthology.orchestra.run.smithi042.stdout:1097 read 0xbd77fe6 thru 0xbd7b87f (0x389a bytes) 2024-09-16T12:59:31.029 INFO:teuthology.orchestra.run.smithi042.stdout:1098 writesame 0xdb14639 thru 0xdb1885f data_size 0x5(0x4227 bytes) 2024-09-16T12:59:31.042 INFO:teuthology.orchestra.run.smithi042.stdout:1100 trunc from 0xe25c49a to 0xa9f8400 2024-09-16T12:59:31.058 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:31.058 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:31.061 INFO:teuthology.orchestra.run.smithi042.stdout:1101 punch from 0xa207030 to 0xa216e3c, (0xfe0c bytes) 2024-09-16T12:59:31.061 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:31.062 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:31.062 INFO:teuthology.orchestra.run.smithi042.stdout:1104 write 0x549bf77 thru 0x54a7e10 (0xbe9a bytes) 2024-09-16T12:59:31.067 INFO:teuthology.orchestra.run.smithi042.stdout:1106 write 0x33d52c4 thru 0x33d71c3 (0x1f00 bytes) 2024-09-16T12:59:31.068 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:31.069 INFO:teuthology.orchestra.run.smithi042.stdout:1107 write 0x6a9aedd thru 0x6a9ba9a (0xbbe bytes) 2024-09-16T12:59:31.071 INFO:teuthology.orchestra.run.smithi042.stdout:1108 punch from 0x5cd0a4f to 0x5cd2195, (0x1746 bytes) 2024-09-16T12:59:31.071 INFO:teuthology.orchestra.run.smithi042.stdout:1109 writesame 0x1be38f4 thru 0x1be40e1 data_size 0x1d(0x7ee bytes) 2024-09-16T12:59:31.081 INFO:teuthology.orchestra.run.smithi042.stdout:1110 trunc from 0xa9f8400 to 0xe8c8404 2024-09-16T12:59:31.089 INFO:teuthology.orchestra.run.smithi042.stdout:1112 writesame 0xc8f977e thru 0xc906f78 data_size 0x1(0xd7fb bytes) 2024-09-16T12:59:31.098 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:31.087+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:31.099 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:31.119 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:31.130 INFO:teuthology.orchestra.run.smithi042.stdout:1113 write 0xb4f7b15 thru 0xb506c48 (0xf134 bytes) 2024-09-16T12:59:31.135 INFO:teuthology.orchestra.run.smithi042.stdout:1114 read 0x125ce2e thru 0x1266678 (0x984b bytes) 2024-09-16T12:59:31.136 INFO:teuthology.orchestra.run.smithi042.stdout:1115 clone 8 order 21 su 1048576 sc 6 2024-09-16T12:59:31.142 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:31.159 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:31.165 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:31.165 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:31.174 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:31.177 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:31.194 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:31.198 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:31.191+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:31.215 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:31.237 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:31.239 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:31.239 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:31.239 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:31.239 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:31.239 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:31.239 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:31.239 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:31.239 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:31.275 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T12:59:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.640792310000002, 2024-09-16T12:59:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006297965, 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629315+0000", 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7080000000000002e-06 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629324+0000", 2024-09-16T12:59:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0250000000000008e-06 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629340+0000", 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.59e-05 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629351+0000", 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1236999999999999e-05 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629565+0000", 2024-09-16T12:59:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000213482 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629582+0000", 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7482999999999998e-05 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.631980+0000", 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023977830000000001 2024-09-16T12:59:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.632247+0000", 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000266867 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.632356+0000", 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010894 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.633744+0000", 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013876240000000001 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635507+0000", 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001763288 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635562+0000", 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5096999999999998e-05 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635608+0000", 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5531000000000002e-05 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.628726712999999, 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063927960000000001, 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641378+0000", 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6429999999999999e-06 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641390+0000", 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1912e-05 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641422+0000", 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2666999999999997e-05 2024-09-16T12:59:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641448+0000", 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5636e-05 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641623+0000", 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017479299999999999 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641641+0000", 2024-09-16T12:59:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8369999999999999e-05 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.644629+0000", 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00298745 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.644995+0000", 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000366688 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.645348+0000", 2024-09-16T12:59:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035283200000000003 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647159+0000", 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018104760000000001 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647631+0000", 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047253799999999999 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647708+0000", 2024-09-16T12:59:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6715e-05 2024-09-16T12:59:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647768+0000", 2024-09-16T12:59:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0075999999999999e-05 2024-09-16T12:59:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T12:59:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.621146754000002, 2024-09-16T12:59:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006003988, 2024-09-16T12:59:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 7 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648958+0000", 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0280000000000001e-06 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648970+0000", 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1374e-05 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649002+0000", 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2298999999999997e-05 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649014+0000", 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2442e-05 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649099+0000", 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4655999999999997e-05 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649107+0000", 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0800000000000006e-06 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.652487+0000", 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003380241 2024-09-16T12:59:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.652733+0000", 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024601000000000001 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.653091+0000", 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035755599999999998 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654603+0000", 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001512163 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654795+0000", 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000192179 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654875+0000", 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0211999999999997e-05 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654959+0000", 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3747999999999998e-05 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:11 4.1f e34/31 4:fa654f18:::rbd_id.image_client.0:head v 34'1, mlcod=0'0)", 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:13.789305+0000", 2024-09-16T12:59:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.480797168999999, 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051335770000000003, 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789305+0000", 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789305+0000", 2024-09-16T12:59:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789306+0000", 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1850000000000001e-06 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789317+0000", 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1344e-05 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789348+0000", 2024-09-16T12:59:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0545e-05 2024-09-16T12:59:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789363+0000", 2024-09-16T12:59:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.485e-05 2024-09-16T12:59:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789438+0000", 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4884999999999999e-05 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789482+0000", 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4731999999999998e-05 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794352+0000", 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048695079999999998 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794438+0000", 2024-09-16T12:59:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6527999999999996e-05 2024-09-16T12:59:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:439 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.005732+0000", 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.264369584000001, 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050891779999999998, 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 439 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005732+0000", 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005732+0000", 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005734+0000", 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4139999999999999e-06 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005741+0000", 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.503e-06 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005768+0000", 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6982e-05 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.005787+0000", 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8974000000000002e-05 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.007568+0000", 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017811070000000001 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.007851+0000", 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028264799999999998 2024-09-16T12:59:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.008426+0000", 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057548500000000004 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.009913+0000", 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001486376 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.010615+0000", 2024-09-16T12:59:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070175700000000001 2024-09-16T12:59:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.010750+0000", 2024-09-16T12:59:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135685 2024-09-16T12:59:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.010822+0000", 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1247000000000004e-05 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:513 4.8 4:132f6b09:::rbd_data.11b59d02cc38.000000000000008c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.089009393, 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054902010000000001, 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 513 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181095+0000", 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4059999999999999e-06 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181106+0000", 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0868e-05 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181145+0000", 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9168000000000002e-05 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181157+0000", 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2343e-05 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181229+0000", 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1264000000000001e-05 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181415+0000", 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000186804 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181722+0000", 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030624200000000003 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.185844+0000", 2024-09-16T12:59:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041219359999999997 2024-09-16T12:59:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186379+0000", 2024-09-16T12:59:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053546599999999996 2024-09-16T12:59:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186495+0000", 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115575 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186583+0000", 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8128999999999996e-05 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:532 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.219756+0000", 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.050346228, 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051249149999999999, 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 532 2024-09-16T12:59:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219756+0000", 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219756+0000", 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219758+0000", 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3099999999999999e-06 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219770+0000", 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1742999999999999e-05 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219805+0000", 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5102999999999999e-05 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219818+0000", 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.219971+0000", 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015284000000000001 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.220132+0000", 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000161905 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.220456+0000", 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032376300000000002 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.221604+0000", 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001148135 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.224727+0000", 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003122545 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.224827+0000", 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9901000000000005e-05 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.224881+0000", 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3786999999999998e-05 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:542 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.036069770999999, 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060375849999999998, 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 542 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234033+0000", 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8599999999999996e-07 2024-09-16T12:59:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234038+0000", 2024-09-16T12:59:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4830000000000001e-06 2024-09-16T12:59:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234047+0000", 2024-09-16T12:59:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3109999999999995e-06 2024-09-16T12:59:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234053+0000", 2024-09-16T12:59:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6339999999999998e-06 2024-09-16T12:59:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.235355+0000", 2024-09-16T12:59:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013027589999999999 2024-09-16T12:59:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.235617+0000", 2024-09-16T12:59:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026182000000000001 2024-09-16T12:59:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.236547+0000", 2024-09-16T12:59:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093014599999999997 2024-09-16T12:59:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239456+0000", 2024-09-16T12:59:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029083289999999999 2024-09-16T12:59:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239788+0000", 2024-09-16T12:59:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033232900000000002 2024-09-16T12:59:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239946+0000", 2024-09-16T12:59:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015829700000000001 2024-09-16T12:59:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.240070+0000", 2024-09-16T12:59:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012349099999999999 2024-09-16T12:59:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:571 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.961037880999999, 2024-09-16T12:59:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005551762, 2024-09-16T12:59:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 571 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309066+0000", 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2060000000000001e-06 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309080+0000", 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3413e-05 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309114+0000", 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4180000000000001e-05 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309126+0000", 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2129e-05 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309312+0000", 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018575800000000001 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309477+0000", 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000164885 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.310054+0000", 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057696199999999998 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.311809+0000", 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001755047 2024-09-16T12:59:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314482+0000", 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026735330000000001 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314559+0000", 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6884000000000005e-05 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314616+0000", 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6765000000000003e-05 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:589 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.335679+0000", 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.934423163, 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050963889999999998, 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 589 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335679+0000", 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335679+0000", 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335680+0000", 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5700000000000001e-07 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335684+0000", 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6149999999999999e-06 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335697+0000", 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2465e-05 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.335703+0000", 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7870000000000004e-06 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.336921+0000", 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001217625 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.337168+0000", 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024640300000000002 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.337796+0000", 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00062875099999999998 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340375+0000", 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025782579999999999 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340644+0000", 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026920399999999999 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340712+0000", 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639000000000005e-05 2024-09-16T12:59:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.340775+0000", 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2785000000000003e-05 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:638 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.766866318, 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052595139999999999, 2024-09-16T12:59:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 638 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503237+0000", 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.02e-06 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503248+0000", 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1114000000000001e-05 2024-09-16T12:59:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503263+0000", 2024-09-16T12:59:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092999999999999e-05 2024-09-16T12:59:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503273+0000", 2024-09-16T12:59:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0413e-05 2024-09-16T12:59:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.504835+0000", 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001562164 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.505031+0000", 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000195465 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.505731+0000", 2024-09-16T12:59:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070030899999999998 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507931+0000", 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022001429999999999 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508299+0000", 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036765399999999997 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508405+0000", 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106368 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508495+0000", 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9771000000000005e-05 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:746 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.500966572999999, 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052549399999999996, 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 746 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769137+0000", 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330000000000001e-06 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769144+0000", 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2660000000000004e-06 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769162+0000", 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7997000000000001e-05 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769178+0000", 2024-09-16T12:59:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6305999999999999e-05 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.770704+0000", 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015260880000000001 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.770965+0000", 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026027800000000002 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.771962+0000", 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099762399999999991 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.773232+0000", 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012702499999999999 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774136+0000", 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000903355 2024-09-16T12:59:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774275+0000", 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139431 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774390+0000", 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011511200000000001 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:821 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.352054337, 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052264670000000003, 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 821 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918049+0000", 2024-09-16T12:59:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.734e-06 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918058+0000", 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918078+0000", 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0163e-05 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918100+0000", 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1396999999999999e-05 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.919700+0000", 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001600838 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.919878+0000", 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177276 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.920899+0000", 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010218219999999999 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.922776+0000", 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001876436 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923007+0000", 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023069799999999999 2024-09-16T12:59:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923160+0000", 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015317299999999999 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923274+0000", 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114298 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:843 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.961705+0000", 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.308396694000001, 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049882310000000001, 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 843 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961705+0000", 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961705+0000", 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961706+0000", 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6200000000000006e-07 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961710+0000", 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8619999999999996e-06 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961718+0000", 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7999999999999999e-06 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.961723+0000", 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6859999999999997e-06 2024-09-16T12:59:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.963536+0000", 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018133240000000001 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.963693+0000", 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015719 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.964589+0000", 2024-09-16T12:59:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089570199999999996 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.965980+0000", 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001391572 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.966585+0000", 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060418400000000001 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.966646+0000", 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1038999999999996e-05 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.966693+0000", 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.791e-05 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1076 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.433166+0000", 2024-09-16T12:59:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.836936219, 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051147570000000002, 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1076 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433166+0000", 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433166+0000", 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433168+0000", 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.57e-06 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433180+0000", 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2206e-05 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433215+0000", 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4106000000000002e-05 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433228+0000", 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3159999999999999e-05 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433369+0000", 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014099699999999999 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433647+0000", 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027864699999999998 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.434793+0000", 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011460559999999999 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.437959+0000", 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031659980000000002 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.438128+0000", 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000168908 2024-09-16T12:59:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.438232+0000", 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103426 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.438280+0000", 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8683000000000001e-05 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1077 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.836852644, 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062511429999999998, 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1077 2024-09-16T12:59:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433250+0000", 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1370000000000001e-06 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433256+0000", 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433266+0000", 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0529e-05 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433278+0000", 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2363e-05 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.435866+0000", 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025878519999999999 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.436125+0000", 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002587 2024-09-16T12:59:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.436955+0000", 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082961500000000002 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.438686+0000", 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017308930000000001 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439352+0000", 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066689400000000002 2024-09-16T12:59:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439455+0000", 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102876 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439500+0000", 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5151000000000002e-05 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:3528 4.8 4:15914a01:::rbd_data.11b5c378985b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.2521947149999999, 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051798410000000001, 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3528 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017909+0000", 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.283e-06 2024-09-16T12:59:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017915+0000", 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4729999999999997e-06 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017941+0000", 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5910000000000001e-05 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017951+0000", 2024-09-16T12:59:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0557000000000001e-05 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.018552+0000", 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060063100000000002 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.018663+0000", 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011089199999999999 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.023087+0000", 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044240950000000003 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:4787 4.15 4:a80ea28d:::rbd_header.11b5b1b9bb41:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3999063960000004, 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060070260000000004, 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4787 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870199+0000", 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2660000000000002e-06 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870212+0000", 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2794e-05 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870248+0000", 2024-09-16T12:59:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6798999999999998e-05 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870266+0000", 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7288000000000001e-05 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870385+0000", 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119671 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870394+0000", 2024-09-16T12:59:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4300000000000006e-06 2024-09-16T12:59:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.873802+0000", 2024-09-16T12:59:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034079269999999998 2024-09-16T12:59:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.873905+0000", 2024-09-16T12:59:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102895 2024-09-16T12:59:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.874314+0000", 2024-09-16T12:59:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040942099999999998 2024-09-16T12:59:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.875454+0000", 2024-09-16T12:59:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001139784 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876028+0000", 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057454900000000004 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876132+0000", 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010318399999999999 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876203+0000", 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1018000000000003e-05 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5060 4.7 4:e6558db2:::rbd_data.11b517fb51dd.0000000000000002:9 [sparse-read 2097152~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9945016280000001, 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074022819999999996, 2024-09-16T12:59:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5060 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275602+0000", 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1509999999999998e-06 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275611+0000", 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0490000000000004e-06 2024-09-16T12:59:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275632+0000", 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1115999999999999e-05 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275643+0000", 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1394e-05 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.279273+0000", 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003629546 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.279360+0000", 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7213000000000007e-05 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.283003+0000", 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003642813 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5061 4.7 4:e6558db2:::rbd_data.11b517fb51dd.0000000000000002:9 [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9944486000000001, 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073381729999999999, 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5061 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275654+0000", 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.074e-06 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275658+0000", 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6880000000000001e-06 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275668+0000", 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7839999999999998e-06 2024-09-16T12:59:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275674+0000", 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9889999999999997e-06 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282791+0000", 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071176089999999996 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282884+0000", 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2085999999999999e-05 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282992+0000", 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010794300000000001 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.401 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:31.402 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:31.403 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:31.399+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:31.503 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:31.499+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:31.510 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:31.510 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:31.510 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:31.510 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:31.519 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:31.604 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:31.599+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:31.622 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:31.622 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:31.622 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:31.622 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:31.622 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:31.631 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:31.705 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:31.699+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:31.726 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:31.726 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:31.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:31.726 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:31.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.726 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:31.726 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:31.726 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.937195104000001, 2024-09-16T12:59:31.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089374989999999998, 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 11 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785978+0000", 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.373e-06 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785989+0000", 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1596999999999999e-05 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786008+0000", 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8510999999999999e-05 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786016+0000", 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786102+0000", 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6339000000000001e-05 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786113+0000", 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1386000000000001e-05 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.788774+0000", 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026609519999999998 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.788888+0000", 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113189 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789043+0000", 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015570800000000001 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794705+0000", 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056619640000000002 2024-09-16T12:59:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794770+0000", 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4789999999999995e-05 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794853+0000", 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3441000000000003e-05 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794913+0000", 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9157999999999998e-05 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:485 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.614987319000001, 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066273469999999996, 2024-09-16T12:59:31.733 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 485 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108185+0000", 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3700000000000002e-06 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108195+0000", 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2839999999999995e-06 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108220+0000", 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5066999999999999e-05 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108229+0000", 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5019999999999998e-06 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108334+0000", 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105136 2024-09-16T12:59:31.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108571+0000", 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236495 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.109793+0000", 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012221809999999999 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114733+0000", 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049399350000000003 2024-09-16T12:59:31.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114745+0000", 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2228999999999999e-05 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114783+0000", 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7716999999999998e-05 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114810+0000", 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7430999999999999e-05 2024-09-16T12:59:31.738 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:635 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.249584818000001, 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046232310000000002, 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 635 2024-09-16T12:59:31.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473585+0000", 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473588+0000", 2024-09-16T12:59:31.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9529999999999999e-06 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473602+0000", 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3266e-05 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473632+0000", 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0015e-05 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473641+0000", 2024-09-16T12:59:31.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5359999999999998e-06 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473734+0000", 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3286000000000001e-05 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.473875+0000", 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140211 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.474719+0000", 2024-09-16T12:59:31.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084420100000000004 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.477891+0000", 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031721079999999999 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478108+0000", 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021716899999999999 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478167+0000", 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8428999999999999e-05 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.478209+0000", 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2057000000000001e-05 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1248 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'11, mlcod=39'8)", 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.695128234, 2024-09-16T12:59:31.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053616649999999998, 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028044+0000", 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5e-06 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028058+0000", 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4469e-05 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028091+0000", 2024-09-16T12:59:31.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2517000000000001e-05 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028106+0000", 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5116000000000001e-05 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028219+0000", 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113768 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028268+0000", 2024-09-16T12:59:31.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8745000000000002e-05 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033313+0000", 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050452869999999999 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033404+0000", 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0263e-05 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1249 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'12, mlcod=39'8)", 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.029057+0000", 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.694113106, 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044055769999999999, 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029057+0000", 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029057+0000", 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029059+0000", 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5370000000000001e-06 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029070+0000", 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1593000000000001e-05 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029095+0000", 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4505999999999999e-05 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029105+0000", 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7769999999999997e-06 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029214+0000", 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109887 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.029273+0000", 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8709000000000003e-05 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033408+0000", 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041346880000000001 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033463+0000", 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4880000000000003e-05 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1250 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'13, mlcod=39'8)", 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.692807154, 2024-09-16T12:59:31.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061776180000000002, 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030364+0000", 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.356e-06 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030375+0000", 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0053e-05 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030395+0000", 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0767e-05 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030404+0000", 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4430000000000003e-06 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030492+0000", 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7842000000000004e-05 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030514+0000", 2024-09-16T12:59:31.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2819999999999998e-05 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036453+0000", 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059381169999999997 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036541+0000", 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8220000000000006e-05 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1252 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'15, mlcod=39'9)", 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.033997+0000", 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.689172885, 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047268359999999999, 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033997+0000", 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033997+0000", 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033998+0000", 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.054e-06 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034023+0000", 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5041e-05 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034048+0000", 2024-09-16T12:59:31.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4097999999999999e-05 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034058+0000", 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0515e-05 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034128+0000", 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0362000000000002e-05 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.034165+0000", 2024-09-16T12:59:31.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6600999999999997e-05 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.038652+0000", 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044872999999999996 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.038724+0000", 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1865000000000004e-05 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1263 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'19, mlcod=39'15)", 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.047178+0000", 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.675992359, 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045177070000000001, 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047178+0000", 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047178+0000", 2024-09-16T12:59:31.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047179+0000", 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000003e-07 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047197+0000", 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8629000000000001e-05 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047211+0000", 2024-09-16T12:59:31.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3699999999999999e-05 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047218+0000", 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1040000000000001e-06 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047273+0000", 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4483e-05 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.047297+0000", 2024-09-16T12:59:31.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.438e-05 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051620+0000", 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043228920000000001 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.051696+0000", 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5604000000000003e-05 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1296 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'29, mlcod=39'25)", 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.644033627000001, 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004900616, 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079137+0000", 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1800000000000005e-07 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079145+0000", 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3760000000000002e-06 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079159+0000", 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4438e-05 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079168+0000", 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2299999999999997e-06 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079253+0000", 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4686999999999997e-05 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.079299+0000", 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6072000000000002e-05 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084001+0000", 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047020100000000004 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084037+0000", 2024-09-16T12:59:31.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5985000000000002e-05 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1301 4.5 4:a692e91d:::rbd_data.11b53140b009.00000000000000c4:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.642769960000001, 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060951929999999996, 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1301 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080402+0000", 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.75e-06 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080409+0000", 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2219999999999996e-06 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080429+0000", 2024-09-16T12:59:31.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9527000000000001e-05 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080438+0000", 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7819999999999999e-06 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080503+0000", 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5438999999999995e-05 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080693+0000", 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190218 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080909+0000", 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021529000000000001 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084016+0000", 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031074319999999998 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086352+0000", 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023357690000000001 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086446+0000", 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4158000000000002e-05 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086495+0000", 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9605999999999999e-05 2024-09-16T12:59:31.775 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1297 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'30, mlcod=39'27)", 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.081247+0000", 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.641923296, 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050937860000000003, 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081247+0000", 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081247+0000", 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081248+0000", 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7199999999999997e-07 2024-09-16T12:59:31.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081255+0000", 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2169999999999997e-06 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081272+0000", 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6671e-05 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081278+0000", 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4350000000000002e-06 2024-09-16T12:59:31.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081341+0000", 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3001000000000002e-05 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.081365+0000", 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3247e-05 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086290+0000", 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049256860000000003 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086341+0000", 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0556999999999997e-05 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:3618 4.a 4:51b44bfb:::rbd_data.11b5c378985b.0000000000000003:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:22.210677+0000", 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5124932139999991, 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045806279999999998, 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.780 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3618 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210677+0000", 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210677+0000", 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210680+0000", 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8899999999999999e-06 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210699+0000", 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9349999999999999e-05 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210736+0000", 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6631999999999998e-05 2024-09-16T12:59:31.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210750+0000", 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3757000000000001e-05 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.210924+0000", 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017455300000000001 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.211175+0000", 2024-09-16T12:59:31.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025073700000000001 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.211578+0000", 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040305499999999998 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.213135+0000", 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001557206 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.215114+0000", 2024-09-16T12:59:31.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019784210000000002 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.215200+0000", 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6254000000000005e-05 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.215258+0000", 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7772999999999997e-05 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:3722 4.a 4:57b538f8:::rbd_header.11b5d5152bdd:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:22.903924+0000", 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.8192462519999992, 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068689479999999997, 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3722 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903924+0000", 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903924+0000", 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903928+0000", 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5099999999999999e-06 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903944+0000", 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6422999999999998e-05 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903982+0000", 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8025999999999998e-05 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.904003+0000", 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0843e-05 2024-09-16T12:59:31.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.904124+0000", 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012166 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.904135+0000", 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0703999999999999e-05 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.908409+0000", 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042740410000000001 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.908554+0000", 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014520900000000001 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.908771+0000", 2024-09-16T12:59:31.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021657999999999999 2024-09-16T12:59:31.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.909753+0000", 2024-09-16T12:59:31.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0009819430000000001 2024-09-16T12:59:31.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.910682+0000", 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092942399999999998 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.910744+0000", 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1796000000000003e-05 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.910793+0000", 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8789000000000003e-05 2024-09-16T12:59:31.792 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:4074 4.a 4:57b538f8:::rbd_header.11b5d5152bdd:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:23.843397+0000", 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 7.8797732649999999, 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078647490000000007, 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.793 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4074 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843397+0000", 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843397+0000", 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843400+0000", 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5560000000000002e-06 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843409+0000", 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6120000000000005e-06 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843434+0000", 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5125000000000001e-05 2024-09-16T12:59:31.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843448+0000", 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3550000000000001e-05 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843527+0000", 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9359999999999999e-05 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:31.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843534+0000", 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8220000000000001e-06 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.847907+0000", 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043724819999999996 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.848103+0000", 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019643899999999999 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.848776+0000", 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067343899999999996 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.850197+0000", 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014204269999999999 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.850995+0000", 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00079860300000000005 2024-09-16T12:59:31.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.851155+0000", 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159479 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.851262+0000", 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010685500000000001 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3589765429999998, 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T12:59:31.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5076 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.2712588719999998, 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020443647999999998, 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5076 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:31.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451913+0000", 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451920+0000", 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5459999999999997e-06 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451938+0000", 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7581999999999999e-05 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451950+0000", 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2033e-05 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452042+0000", 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2533000000000003e-05 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452251+0000", 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020887 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.456761+0000", 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045099959999999996 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472274+0000", 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015512882 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472281+0000", 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3939999999999998e-06 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472324+0000", 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2389999999999999e-05 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472355+0000", 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1436000000000002e-05 2024-09-16T12:59:31.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5232 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [write 26019337~28881] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.900074+0000", 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8230962079999999, 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058250639999999996, 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5232 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.810 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900074+0000", 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900074+0000", 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900077+0000", 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7700000000000002e-06 2024-09-16T12:59:31.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900113+0000", 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6511999999999999e-05 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900156+0000", 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2483e-05 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900177+0000", 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1192999999999999e-05 2024-09-16T12:59:31.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900300+0000", 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012320699999999999 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900310+0000", 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6099999999999995e-06 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.903745+0000", 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034353640000000002 2024-09-16T12:59:31.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.903815+0000", 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9387999999999994e-05 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.903903+0000", 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8127999999999994e-05 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.904766+0000", 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086280999999999999 2024-09-16T12:59:31.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.905841+0000", 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00107515 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.905877+0000", 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6720000000000001e-05 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.905899+0000", 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1729000000000002e-05 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5513 4.13 e46/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 46'80, mlcod=46'78)", 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.187958633, 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010685616, 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535213+0000", 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1060000000000001e-06 2024-09-16T12:59:31.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535222+0000", 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0159999999999997e-06 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535240+0000", 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7926000000000001e-05 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535247+0000", 2024-09-16T12:59:31.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5820000000000004e-06 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535345+0000", 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7705000000000001e-05 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535395+0000", 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9996e-05 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.545844+0000", 2024-09-16T12:59:31.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010448687999999999 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.545897+0000", 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3597000000000002e-05 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5580 4.12 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.0301358139999999, 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034176870999999998, 2024-09-16T12:59:31.820 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5580 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693037+0000", 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0650000000000001e-06 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693045+0000", 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693070+0000", 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5341999999999999e-05 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693081+0000", 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0611000000000001e-05 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693182+0000", 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100437 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693309+0000", 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012741400000000001 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.703090+0000", 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097814800000000004 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726908+0000", 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023817680000000001 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727023+0000", 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011449400000000001 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727130+0000", 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010704099999999999 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727211+0000", 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1675000000000007e-05 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.825 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:5580 4.12 e46/40 ondisk, result = 0)", 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.718821+0000", 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.004349012, 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084041510000000003, 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718821+0000", 2024-09-16T12:59:31.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718821+0000", 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718823+0000", 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8190000000000001e-06 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:31.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718832+0000", 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4549999999999995e-06 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718861+0000", 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9263000000000001e-05 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718873+0000", 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1773999999999999e-05 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:31.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726990+0000", 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081170989999999991 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727007+0000", 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7416000000000001e-05 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727225+0000", 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021832500000000001 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:31.830 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:31.830 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:31.831 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:31.831 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:31.823+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T12:59:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T12:59:31.931 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:31.923+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:31.936 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T12:59:31.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T12:59:32.000 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:32.000 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:32.000 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:32.000 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:32.010 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:32.032 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:32.027+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:32.108 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:32.108 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:32.108 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:32.108 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:32.108 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:32.118 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:32.133 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:32.123+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:32.234 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:32.231+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:32.260 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.706479962000003, 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T12:59:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T12:59:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T12:59:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.881653378000003, 2024-09-16T12:59:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010809278, 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374549+0000", 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.037e-06 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374711+0000", 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016226599999999999 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374734+0000", 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2895999999999999e-05 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374746+0000", 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.221e-05 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.380942+0000", 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061959390000000001 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381063+0000", 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121274 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381255+0000", 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019110500000000001 2024-09-16T12:59:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384487+0000", 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032327979999999998 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385296+0000", 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080888499999999996 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385332+0000", 2024-09-16T12:59:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5774e-05 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385357+0000", 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5094000000000001e-05 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.881442485999997, 2024-09-16T12:59:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010976025, 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 45 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374760+0000", 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.13e-06 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374769+0000", 2024-09-16T12:59:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5699999999999999e-06 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374819+0000", 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9882000000000001e-05 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374828+0000", 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382329+0000", 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075009009999999999 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382403+0000", 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4208999999999994e-05 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382514+0000", 2024-09-16T12:59:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111276 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385533+0000", 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030190080000000001 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385564+0000", 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0926000000000002e-05 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385666+0000", 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101791 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385735+0000", 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8541999999999995e-05 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:32.279 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.881362914999997, 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:32.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T12:59:32.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T12:59:32.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T12:59:32.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T12:59:32.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T12:59:32.289 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.289 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.289 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.289 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.881315659000002, 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T12:59:32.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T12:59:32.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T12:59:32.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T12:59:32.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:32.296 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.881112723000001, 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T12:59:32.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T12:59:32.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:32.302 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.881081461000001, 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:32.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T12:59:32.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T12:59:32.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.308 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.881045698999998, 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T12:59:32.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.880851069000002, 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T12:59:32.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T12:59:32.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T12:59:32.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.880804224000002, 2024-09-16T12:59:32.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T12:59:32.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T12:59:32.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.336744183, 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010623719, 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.326 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 69 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919458+0000", 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.474e-06 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919621+0000", 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016234800000000001 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919641+0000", 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0336999999999999e-05 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919656+0000", 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4554e-05 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924273+0000", 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046169610000000002 2024-09-16T12:59:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924411+0000", 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138286 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924627+0000", 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021597399999999999 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928794+0000", 2024-09-16T12:59:32.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004167268 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929983+0000", 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011884409999999999 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930039+0000", 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6363000000000002e-05 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930081+0000", 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1712999999999999e-05 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.336528053000002, 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.332 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T12:59:32.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.337 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.336474146, 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T12:59:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T12:59:32.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T12:59:32.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.336431841, 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.344 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T12:59:32.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T12:59:32.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.349 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.336273415999997, 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.350 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T12:59:32.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.336223935, 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T12:59:32.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T12:59:32.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T12:59:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.336093364999996, 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T12:59:32.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T12:59:32.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.335911852000002, 2024-09-16T12:59:32.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T12:59:32.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T12:59:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.525384185, 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.373 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:32.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T12:59:32.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.525235231000003, 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:32.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:32.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T12:59:32.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:32.385 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:32.386 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-16T12:59:32.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:32.389 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:59:32.390 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T12:59:32.390 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-16T12:59:32.244364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":143850633,"num_objects":131,"num_object_clones":25,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":5,"num_read":11112,"num_read_kb":30592,"num_write":2033,"num_write_kb":55681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":55544,"num_keys_recovered":19,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1590,"ondisk_log_size":1590,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2122328,"kb_used_data":7552,"kb_used_omap":11,"kb_used_meta":2114676,"kb_avail":658480552,"statfs":{"total":676457349120,"available":674284085248,"internally_reserved":0,"allocated":7733248,"data_stored":40488504,"data_compressed":450031,"data_compressed_allocated":3059712,"data_compressed_original":38256890,"omap_allocated":11928,"internal_metadata":2165428584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39130300,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":3698,"num_read_kb":11958,"num_write":639,"num_write_kb":36792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001115"},"pg_stats":[{"pgid":"4.1f","version":"41'14","reported_seq":176,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.907151+0000","last_change":"2024-09-16T12:59:13.789786+0000","last_active":"2024-09-16T12:59:28.907151+0000","last_peered":"2024-09-16T12:59:28.907151+0000","last_clean":"2024-09-16T12:59:28.907151+0000","last_became_active":"2024-09-16T12:59:11.766798+0000","last_became_peered":"2024-09-16T12:59:11.766798+0000","last_unstale":"2024-09-16T12:59:28.907151+0000","last_undegraded":"2024-09-16T12:59:28.907151+0000","last_fullsized":"2024-09-16T12:59:28.907151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:22.588888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":1386038,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":142,"num_read_kb":1321,"num_write":27,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"47'61","reported_seq":128,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:29.739657+0000","last_change":"2024-09-16T12:59:14.030480+0000","last_active":"2024-09-16T12:59:29.739657+0000","last_peered":"2024-09-16T12:59:29.739657+0000","last_clean":"2024-09-16T12:59:29.739657+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T12:59:29.739657+0000","last_undegraded":"2024-09-16T12:59:29.739657+0000","last_fullsized":"2024-09-16T12:59:29.739657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049602200000000004,"stat_sum":{"num_bytes":12039604,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":81,"num_read_kb":787,"num_write":72,"num_write_kb":1212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"38'6","reported_seq":51,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.891885+0000","last_change":"2024-09-16T12:59:20.858214+0000","last_active":"2024-09-16T12:59:28.891885+0000","last_peered":"2024-09-16T12:59:28.891885+0000","last_clean":"2024-09-16T12:59:28.891885+0000","last_became_active":"2024-09-16T12:59:20.837194+0000","last_became_peered":"2024-09-16T12:59:20.837194+0000","last_unstale":"2024-09-16T12:59:28.891885+0000","last_undegraded":"2024-09-16T12:59:28.891885+0000","last_fullsized":"2024-09-16T12:59:28.891885+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:59.019644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012332339999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"45'47","reported_seq":147,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.883759+0000","last_change":"2024-09-16T12:59:20.868824+0000","last_active":"2024-09-16T12:59:28.883759+0000","last_peered":"2024-09-16T12:59:28.883759+0000","last_clean":"2024-09-16T12:59:28.883759+0000","last_became_active":"2024-09-16T12:59:20.837393+0000","last_became_peered":"2024-09-16T12:59:20.837393+0000","last_unstale":"2024-09-16T12:59:28.883759+0000","last_undegraded":"2024-09-16T12:59:28.883759+0000","last_fullsized":"2024-09-16T12:59:28.883759+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:51:34.761291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015138090000000001,"stat_sum":{"num_bytes":1059932,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":127,"num_read_kb":198,"num_write":55,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"41'9","reported_seq":42,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.885826+0000","last_change":"2024-09-16T12:59:13.779899+0000","last_active":"2024-09-16T12:59:28.885826+0000","last_peered":"2024-09-16T12:59:28.885826+0000","last_clean":"2024-09-16T12:59:28.885826+0000","last_became_active":"2024-09-16T12:59:11.759998+0000","last_became_peered":"2024-09-16T12:59:11.759998+0000","last_unstale":"2024-09-16T12:59:28.885826+0000","last_undegraded":"2024-09-16T12:59:28.885826+0000","last_fullsized":"2024-09-16T12:59:28.885826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:49:22.613940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002573915,"stat_sum":{"num_bytes":1867776,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":49,"num_write":10,"num_write_kb":1723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"44'46","reported_seq":559,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.221652+0000","last_change":"2024-09-16T12:59:13.779650+0000","last_active":"2024-09-16T12:59:30.221652+0000","last_peered":"2024-09-16T12:59:30.221652+0000","last_clean":"2024-09-16T12:59:30.221652+0000","last_became_active":"2024-09-16T12:59:11.759751+0000","last_became_peered":"2024-09-16T12:59:11.759751+0000","last_unstale":"2024-09-16T12:59:30.221652+0000","last_undegraded":"2024-09-16T12:59:30.221652+0000","last_fullsized":"2024-09-16T12:59:30.221652+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:12.049622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002565964,"stat_sum":{"num_bytes":32014824,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":512,"num_read_kb":4094,"num_write":45,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"41'15","reported_seq":49,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.891606+0000","last_change":"2024-09-16T12:59:13.770391+0000","last_active":"2024-09-16T12:59:28.891606+0000","last_peered":"2024-09-16T12:59:28.891606+0000","last_clean":"2024-09-16T12:59:28.891606+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T12:59:28.891606+0000","last_undegraded":"2024-09-16T12:59:28.891606+0000","last_fullsized":"2024-09-16T12:59:28.891606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013782849999999999,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":60,"num_write":20,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"47'67","reported_seq":228,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:29.541482+0000","last_change":"2024-09-16T12:59:13.777900+0000","last_active":"2024-09-16T12:59:29.541482+0000","last_peered":"2024-09-16T12:59:29.541482+0000","last_clean":"2024-09-16T12:59:29.541482+0000","last_became_active":"2024-09-16T12:59:11.767308+0000","last_became_peered":"2024-09-16T12:59:11.767308+0000","last_unstale":"2024-09-16T12:59:29.541482+0000","last_undegraded":"2024-09-16T12:59:29.541482+0000","last_fullsized":"2024-09-16T12:59:29.541482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:40.593369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093274999999999999,"stat_sum":{"num_bytes":5219078,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":404,"num_read_kb":1229,"num_write":110,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"45'17","reported_seq":175,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.892114+0000","last_change":"2024-09-16T12:59:13.770695+0000","last_active":"2024-09-16T12:59:28.892114+0000","last_peered":"2024-09-16T12:59:28.892114+0000","last_clean":"2024-09-16T12:59:28.892114+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T12:59:28.892114+0000","last_undegraded":"2024-09-16T12:59:28.892114+0000","last_fullsized":"2024-09-16T12:59:28.892114+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001481132,"stat_sum":{"num_bytes":5289573,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":133,"num_read_kb":507,"num_write":18,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"44'53","reported_seq":458,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.882816+0000","last_change":"2024-09-16T12:59:14.030762+0000","last_active":"2024-09-16T12:59:28.882816+0000","last_peered":"2024-09-16T12:59:28.882816+0000","last_clean":"2024-09-16T12:59:28.882816+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T12:59:28.882816+0000","last_undegraded":"2024-09-16T12:59:28.882816+0000","last_fullsized":"2024-09-16T12:59:28.882816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685533,"stat_sum":{"num_bytes":11165696,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":883,"num_read_kb":1173,"num_write":49,"num_write_kb":476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"48'90","reported_seq":335,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.165122+0000","last_change":"2024-09-16T12:59:13.796422+0000","last_active":"2024-09-16T12:59:30.165122+0000","last_peered":"2024-09-16T12:59:30.165122+0000","last_clean":"2024-09-16T12:59:30.165122+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T12:59:30.165122+0000","last_undegraded":"2024-09-16T12:59:30.165122+0000","last_fullsized":"2024-09-16T12:59:30.165122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055065799999999999,"stat_sum":{"num_bytes":12615862,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":569,"num_read_kb":1237,"num_write":108,"num_write_kb":695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"45'46","reported_seq":322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.891845+0000","last_change":"2024-09-16T12:59:20.883767+0000","last_active":"2024-09-16T12:59:28.891845+0000","last_peered":"2024-09-16T12:59:28.891845+0000","last_clean":"2024-09-16T12:59:28.891845+0000","last_became_active":"2024-09-16T12:59:20.842644+0000","last_became_peered":"2024-09-16T12:59:20.842644+0000","last_unstale":"2024-09-16T12:59:28.891845+0000","last_undegraded":"2024-09-16T12:59:28.891845+0000","last_fullsized":"2024-09-16T12:59:28.891845+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:18.381117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013482349999999999,"stat_sum":{"num_bytes":4562960,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":676,"num_read_kb":2189,"num_write":90,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"48'98","reported_seq":298,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.078097+0000","last_change":"2024-09-16T12:59:13.780086+0000","last_active":"2024-09-16T12:59:30.078097+0000","last_peered":"2024-09-16T12:59:30.078097+0000","last_clean":"2024-09-16T12:59:30.078097+0000","last_became_active":"2024-09-16T12:59:11.770883+0000","last_became_peered":"2024-09-16T12:59:11.770883+0000","last_unstale":"2024-09-16T12:59:30.078097+0000","last_undegraded":"2024-09-16T12:59:30.078097+0000","last_fullsized":"2024-09-16T12:59:30.078097+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:22.020502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024518000000000001,"stat_sum":{"num_bytes":17123005,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":471,"num_read_kb":803,"num_write":133,"num_write_kb":33479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"48'24","reported_seq":200,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:29.911060+0000","last_change":"2024-09-16T12:59:20.866963+0000","last_active":"2024-09-16T12:59:29.911060+0000","last_peered":"2024-09-16T12:59:29.911060+0000","last_clean":"2024-09-16T12:59:29.911060+0000","last_became_active":"2024-09-16T12:59:20.844338+0000","last_became_peered":"2024-09-16T12:59:20.844338+0000","last_unstale":"2024-09-16T12:59:29.911060+0000","last_undegraded":"2024-09-16T12:59:29.911060+0000","last_fullsized":"2024-09-16T12:59:29.911060+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:17.657544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061900799999999999,"stat_sum":{"num_bytes":559892,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":135,"num_read_kb":259,"num_write":23,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"43'10","reported_seq":46,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.888571+0000","last_change":"2024-09-16T12:59:13.770311+0000","last_active":"2024-09-16T12:59:28.888571+0000","last_peered":"2024-09-16T12:59:28.888571+0000","last_clean":"2024-09-16T12:59:28.888571+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T12:59:28.888571+0000","last_undegraded":"2024-09-16T12:59:28.888571+0000","last_fullsized":"2024-09-16T12:59:28.888571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413508,"stat_sum":{"num_bytes":376877,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":455,"num_write":9,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":37,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.888437+0000","last_change":"2024-09-16T12:59:20.835494+0000","last_active":"2024-09-16T12:59:28.888437+0000","last_peered":"2024-09-16T12:59:28.888437+0000","last_clean":"2024-09-16T12:59:28.888437+0000","last_became_active":"2024-09-16T12:59:20.835160+0000","last_became_peered":"2024-09-16T12:59:20.835160+0000","last_unstale":"2024-09-16T12:59:28.888437+0000","last_undegraded":"2024-09-16T12:59:28.888437+0000","last_fullsized":"2024-09-16T12:59:28.888437+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:42:23.395626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047952,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.907284+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T12:59:28.907284+0000","last_peered":"2024-09-16T12:59:28.907284+0000","last_clean":"2024-09-16T12:59:28.907284+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T12:59:28.907284+0000","last_undegraded":"2024-09-16T12:59:28.907284+0000","last_fullsized":"2024-09-16T12:59:28.907284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"48'35","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.001833+0000","last_change":"2024-09-16T12:59:13.777645+0000","last_active":"2024-09-16T12:59:30.001833+0000","last_peered":"2024-09-16T12:59:30.001833+0000","last_clean":"2024-09-16T12:59:30.001833+0000","last_became_active":"2024-09-16T12:59:11.770325+0000","last_became_peered":"2024-09-16T12:59:11.770325+0000","last_unstale":"2024-09-16T12:59:30.001833+0000","last_undegraded":"2024-09-16T12:59:30.001833+0000","last_fullsized":"2024-09-16T12:59:30.001833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:06:31.319709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069048899999999997,"stat_sum":{"num_bytes":36,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":79,"num_write":34,"num_write_kb":689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.907049+0000","last_change":"2024-09-16T12:59:20.842195+0000","last_active":"2024-09-16T12:59:28.907049+0000","last_peered":"2024-09-16T12:59:28.907049+0000","last_clean":"2024-09-16T12:59:28.907049+0000","last_became_active":"2024-09-16T12:59:20.841956+0000","last_became_peered":"2024-09-16T12:59:20.841956+0000","last_unstale":"2024-09-16T12:59:28.907049+0000","last_undegraded":"2024-09-16T12:59:28.907049+0000","last_fullsized":"2024-09-16T12:59:28.907049+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:53:07.991519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"48'128","reported_seq":698,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.266386+0000","last_change":"2024-09-16T12:59:13.769781+0000","last_active":"2024-09-16T12:59:30.266386+0000","last_peered":"2024-09-16T12:59:30.266386+0000","last_clean":"2024-09-16T12:59:30.266386+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T12:59:30.266386+0000","last_undegraded":"2024-09-16T12:59:30.266386+0000","last_fullsized":"2024-09-16T12:59:30.266386+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010481990000000001,"stat_sum":{"num_bytes":137,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1612,"num_read_kb":1394,"num_write":147,"num_write_kb":364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T12:59:27.896946+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T12:59:27.896946+0000","last_peered":"2024-09-16T12:59:27.896946+0000","last_clean":"2024-09-16T12:59:27.896946+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T12:59:27.896946+0000","last_undegraded":"2024-09-16T12:59:27.896946+0000","last_fullsized":"2024-09-16T12:59:27.896946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"42'19","reported_seq":182,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.883017+0000","last_change":"2024-09-16T12:59:14.030767+0000","last_active":"2024-09-16T12:59:28.883017+0000","last_peered":"2024-09-16T12:59:28.883017+0000","last_clean":"2024-09-16T12:59:28.883017+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T12:59:28.883017+0000","last_undegraded":"2024-09-16T12:59:28.883017+0000","last_fullsized":"2024-09-16T12:59:28.883017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000670212,"stat_sum":{"num_bytes":1807886,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":139,"num_read_kb":1266,"num_write":24,"num_write_kb":623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.882713+0000","last_change":"2024-09-16T12:59:20.840516+0000","last_active":"2024-09-16T12:59:28.882713+0000","last_peered":"2024-09-16T12:59:28.882713+0000","last_clean":"2024-09-16T12:59:28.882713+0000","last_became_active":"2024-09-16T12:59:20.840106+0000","last_became_peered":"2024-09-16T12:59:20.840106+0000","last_unstale":"2024-09-16T12:59:28.882713+0000","last_undegraded":"2024-09-16T12:59:28.882713+0000","last_fullsized":"2024-09-16T12:59:28.882713+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:27.770889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"42'32","reported_seq":192,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.896119+0000","last_change":"2024-09-16T12:59:20.878776+0000","last_active":"2024-09-16T12:59:28.896119+0000","last_peered":"2024-09-16T12:59:28.896119+0000","last_clean":"2024-09-16T12:59:28.896119+0000","last_became_active":"2024-09-16T12:59:20.845288+0000","last_became_peered":"2024-09-16T12:59:20.845288+0000","last_unstale":"2024-09-16T12:59:28.896119+0000","last_undegraded":"2024-09-16T12:59:28.896119+0000","last_fullsized":"2024-09-16T12:59:28.896119+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:11.492412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045226000000000001,"stat_sum":{"num_bytes":1327120,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":348,"num_read_kb":737,"num_write":27,"num_write_kb":337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8208,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.885925+0000","last_change":"2024-09-16T12:59:01.688552+0000","last_active":"2024-09-16T12:59:28.885925+0000","last_peered":"2024-09-16T12:59:28.885925+0000","last_clean":"2024-09-16T12:59:28.885925+0000","last_became_active":"2024-09-16T12:59:01.688068+0000","last_became_peered":"2024-09-16T12:59:01.688068+0000","last_unstale":"2024-09-16T12:59:28.885925+0000","last_undegraded":"2024-09-16T12:59:28.885925+0000","last_fullsized":"2024-09-16T12:59:28.885925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:01.298345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":54,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.896034+0000","last_change":"2024-09-16T12:59:20.882735+0000","last_active":"2024-09-16T12:59:28.896034+0000","last_peered":"2024-09-16T12:59:28.896034+0000","last_clean":"2024-09-16T12:59:28.896034+0000","last_became_active":"2024-09-16T12:59:20.845568+0000","last_became_peered":"2024-09-16T12:59:20.845568+0000","last_unstale":"2024-09-16T12:59:28.896034+0000","last_undegraded":"2024-09-16T12:59:28.896034+0000","last_fullsized":"2024-09-16T12:59:28.896034+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:18.023864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"41'18","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.883698+0000","last_change":"2024-09-16T12:59:20.849504+0000","last_active":"2024-09-16T12:59:28.883698+0000","last_peered":"2024-09-16T12:59:28.883698+0000","last_clean":"2024-09-16T12:59:28.883698+0000","last_became_active":"2024-09-16T12:59:20.832759+0000","last_became_peered":"2024-09-16T12:59:20.832759+0000","last_unstale":"2024-09-16T12:59:28.883698+0000","last_undegraded":"2024-09-16T12:59:28.883698+0000","last_fullsized":"2024-09-16T12:59:28.883698+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:23.812603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077242,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":37,"num_write":17,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.883323+0000","last_change":"2024-09-16T12:59:20.835231+0000","last_active":"2024-09-16T12:59:28.883323+0000","last_peered":"2024-09-16T12:59:28.883323+0000","last_clean":"2024-09-16T12:59:28.883323+0000","last_became_active":"2024-09-16T12:59:20.834767+0000","last_became_peered":"2024-09-16T12:59:20.834767+0000","last_unstale":"2024-09-16T12:59:28.883323+0000","last_undegraded":"2024-09-16T12:59:28.883323+0000","last_fullsized":"2024-09-16T12:59:28.883323+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:28.665721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.883486+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T12:59:28.883486+0000","last_peered":"2024-09-16T12:59:28.883486+0000","last_clean":"2024-09-16T12:59:28.883486+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T12:59:28.883486+0000","last_undegraded":"2024-09-16T12:59:28.883486+0000","last_fullsized":"2024-09-16T12:59:28.883486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"44'19","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.896073+0000","last_change":"2024-09-16T12:59:20.852417+0000","last_active":"2024-09-16T12:59:28.896073+0000","last_peered":"2024-09-16T12:59:28.896073+0000","last_clean":"2024-09-16T12:59:28.896073+0000","last_became_active":"2024-09-16T12:59:20.840157+0000","last_became_peered":"2024-09-16T12:59:20.840157+0000","last_unstale":"2024-09-16T12:59:28.896073+0000","last_undegraded":"2024-09-16T12:59:28.896073+0000","last_fullsized":"2024-09-16T12:59:28.896073+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:14.383652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064957100000000005,"stat_sum":{"num_bytes":10633216,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":31,"num_read_kb":4226,"num_write":18,"num_write_kb":4239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.882581+0000","last_change":"2024-09-16T12:59:01.686741+0000","last_active":"2024-09-16T12:59:28.882581+0000","last_peered":"2024-09-16T12:59:28.882581+0000","last_clean":"2024-09-16T12:59:28.882581+0000","last_became_active":"2024-09-16T12:59:01.686267+0000","last_became_peered":"2024-09-16T12:59:01.686267+0000","last_unstale":"2024-09-16T12:59:28.882581+0000","last_undegraded":"2024-09-16T12:59:28.882581+0000","last_fullsized":"2024-09-16T12:59:28.882581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:49.310418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T12:59:27.897055+0000","last_change":"2024-09-16T12:59:20.840160+0000","last_active":"2024-09-16T12:59:27.897055+0000","last_peered":"2024-09-16T12:59:27.897055+0000","last_clean":"2024-09-16T12:59:27.897055+0000","last_became_active":"2024-09-16T12:59:20.839853+0000","last_became_peered":"2024-09-16T12:59:20.839853+0000","last_unstale":"2024-09-16T12:59:27.897055+0000","last_undegraded":"2024-09-16T12:59:27.897055+0000","last_fullsized":"2024-09-16T12:59:27.897055+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:45.235735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"41'6","reported_seq":179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.882966+0000","last_change":"2024-09-16T12:59:20.845862+0000","last_active":"2024-09-16T12:59:28.882966+0000","last_peered":"2024-09-16T12:59:28.882966+0000","last_clean":"2024-09-16T12:59:28.882966+0000","last_became_active":"2024-09-16T12:59:20.839259+0000","last_became_peered":"2024-09-16T12:59:20.839259+0000","last_unstale":"2024-09-16T12:59:28.882966+0000","last_undegraded":"2024-09-16T12:59:28.882966+0000","last_fullsized":"2024-09-16T12:59:28.882966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:26:22.372402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094804399999999999,"stat_sum":{"num_bytes":1194197,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":1278,"num_write":5,"num_write_kb":255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":30933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.891643+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T12:59:28.891643+0000","last_peered":"2024-09-16T12:59:28.891643+0000","last_clean":"2024-09-16T12:59:28.891643+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T12:59:28.891643+0000","last_undegraded":"2024-09-16T12:59:28.891643+0000","last_fullsized":"2024-09-16T12:59:28.891643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:20.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":40,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.896040+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T12:59:28.896040+0000","last_peered":"2024-09-16T12:59:28.896040+0000","last_clean":"2024-09-16T12:59:28.896040+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T12:59:28.896040+0000","last_undegraded":"2024-09-16T12:59:28.896040+0000","last_fullsized":"2024-09-16T12:59:28.896040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":127,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T12:59:27.896607+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T12:59:27.896607+0000","last_peered":"2024-09-16T12:59:27.896607+0000","last_clean":"2024-09-16T12:59:27.896607+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T12:59:27.896607+0000","last_undegraded":"2024-09-16T12:59:27.896607+0000","last_fullsized":"2024-09-16T12:59:27.896607+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"45'16","reported_seq":187,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.906952+0000","last_change":"2024-09-16T12:59:13.789075+0000","last_active":"2024-09-16T12:59:28.906952+0000","last_peered":"2024-09-16T12:59:28.906952+0000","last_clean":"2024-09-16T12:59:28.906952+0000","last_became_active":"2024-09-16T12:59:11.766779+0000","last_became_peered":"2024-09-16T12:59:11.766779+0000","last_unstale":"2024-09-16T12:59:28.906952+0000","last_undegraded":"2024-09-16T12:59:28.906952+0000","last_fullsized":"2024-09-16T12:59:28.906952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:47.829432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046786199999999999,"stat_sum":{"num_bytes":1209022,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":158,"num_read_kb":344,"num_write":15,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"42'166","reported_seq":371,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.895958+0000","last_change":"2024-09-16T12:59:13.796421+0000","last_active":"2024-09-16T12:59:28.895958+0000","last_peered":"2024-09-16T12:59:28.895958+0000","last_clean":"2024-09-16T12:59:28.895958+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T12:59:28.895958+0000","last_undegraded":"2024-09-16T12:59:28.895958+0000","last_fullsized":"2024-09-16T12:59:28.895958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067313300000000002,"stat_sum":{"num_bytes":5512502,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":939,"num_read_kb":2071,"num_write":264,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"42'50","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.885567+0000","last_change":"2024-09-16T12:59:20.859488+0000","last_active":"2024-09-16T12:59:28.885567+0000","last_peered":"2024-09-16T12:59:28.885567+0000","last_clean":"2024-09-16T12:59:28.885567+0000","last_became_active":"2024-09-16T12:59:20.835273+0000","last_became_peered":"2024-09-16T12:59:20.835273+0000","last_unstale":"2024-09-16T12:59:28.885567+0000","last_undegraded":"2024-09-16T12:59:28.885567+0000","last_fullsized":"2024-09-16T12:59:28.885567+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:17:56.825719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011000280000000001,"stat_sum":{"num_bytes":5144612,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":188,"num_write":72,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"48'61","reported_seq":265,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.212468+0000","last_change":"2024-09-16T12:59:20.859192+0000","last_active":"2024-09-16T12:59:30.212468+0000","last_peered":"2024-09-16T12:59:30.212468+0000","last_clean":"2024-09-16T12:59:30.212468+0000","last_became_active":"2024-09-16T12:59:20.842251+0000","last_became_peered":"2024-09-16T12:59:20.842251+0000","last_unstale":"2024-09-16T12:59:30.212468+0000","last_undegraded":"2024-09-16T12:59:30.212468+0000","last_fullsized":"2024-09-16T12:59:30.212468+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:24.239482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078078700000000002,"stat_sum":{"num_bytes":5136384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":480,"num_read_kb":519,"num_write":56,"num_write_kb":527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8192,"num_keys_recovered":2,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"45'15","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.882635+0000","last_change":"2024-09-16T12:59:20.864100+0000","last_active":"2024-09-16T12:59:28.882635+0000","last_peered":"2024-09-16T12:59:28.882635+0000","last_clean":"2024-09-16T12:59:28.882635+0000","last_became_active":"2024-09-16T12:59:20.842488+0000","last_became_peered":"2024-09-16T12:59:20.842488+0000","last_unstale":"2024-09-16T12:59:28.882635+0000","last_undegraded":"2024-09-16T12:59:28.882635+0000","last_fullsized":"2024-09-16T12:59:28.882635+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:22.250517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053245600000000005,"stat_sum":{"num_bytes":55,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":69,"num_write":20,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"39'32","reported_seq":288,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.885922+0000","last_change":"2024-09-16T12:59:13.780281+0000","last_active":"2024-09-16T12:59:28.885922+0000","last_peered":"2024-09-16T12:59:28.885922+0000","last_clean":"2024-09-16T12:59:28.885922+0000","last_became_active":"2024-09-16T12:59:11.764080+0000","last_became_peered":"2024-09-16T12:59:11.764080+0000","last_unstale":"2024-09-16T12:59:28.885922+0000","last_undegraded":"2024-09-16T12:59:28.885922+0000","last_fullsized":"2024-09-16T12:59:28.885922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:55:18.557326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020639640000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":688,"num_read_kb":600,"num_write":28,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"43'18","reported_seq":193,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.886039+0000","last_change":"2024-09-16T12:59:13.780488+0000","last_active":"2024-09-16T12:59:28.886039+0000","last_peered":"2024-09-16T12:59:28.886039+0000","last_clean":"2024-09-16T12:59:28.886039+0000","last_became_active":"2024-09-16T12:59:11.764699+0000","last_became_peered":"2024-09-16T12:59:11.764699+0000","last_unstale":"2024-09-16T12:59:28.886039+0000","last_undegraded":"2024-09-16T12:59:28.886039+0000","last_fullsized":"2024-09-16T12:59:28.886039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:12.151943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001758094,"stat_sum":{"num_bytes":1410023,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":601,"num_write":18,"num_write_kb":228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"48'287","reported_seq":689,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.258595+0000","last_change":"2024-09-16T12:59:13.780729+0000","last_active":"2024-09-16T12:59:30.258595+0000","last_peered":"2024-09-16T12:59:30.258595+0000","last_clean":"2024-09-16T12:59:30.258595+0000","last_became_active":"2024-09-16T12:59:11.770833+0000","last_became_peered":"2024-09-16T12:59:11.770833+0000","last_unstale":"2024-09-16T12:59:30.258595+0000","last_undegraded":"2024-09-16T12:59:30.258595+0000","last_fullsized":"2024-09-16T12:59:30.258595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:56.784122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637831,"stat_sum":{"num_bytes":2056227,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1929,"num_read_kb":2703,"num_write":399,"num_write_kb":1248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"41'10","reported_seq":52,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.891872+0000","last_change":"2024-09-16T12:59:20.880782+0000","last_active":"2024-09-16T12:59:28.891872+0000","last_peered":"2024-09-16T12:59:28.891872+0000","last_clean":"2024-09-16T12:59:28.891872+0000","last_became_active":"2024-09-16T12:59:20.839661+0000","last_became_peered":"2024-09-16T12:59:20.839661+0000","last_unstale":"2024-09-16T12:59:28.891872+0000","last_undegraded":"2024-09-16T12:59:28.891872+0000","last_fullsized":"2024-09-16T12:59:28.891872+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:11.767806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053943399999999999,"stat_sum":{"num_bytes":647168,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":64,"num_write":11,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":143260246,"num_objects":127,"num_object_clones":25,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":5,"num_read":11046,"num_read_kb":30537,"num_write":1930,"num_write_kb":54051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":55525,"num_keys_recovered":19,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3440640,"data_stored":30032522,"data_compressed":397082,"data_compressed_allocated":2506752,"data_compressed_original":29443010,"omap_allocated":0,"internal_metadata":0},"log_size":1515,"ondisk_log_size":1515,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302384,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069456,"statfs":{"total":96636764160,"available":96327122944,"internally_reserved":0,"allocated":892928,"data_stored":2784457,"data_compressed":29462,"data_compressed_allocated":221184,"data_compressed_original":2485123,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302636,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069204,"statfs":{"total":96636764160,"available":96326864896,"internally_reserved":0,"allocated":1142784,"data_stored":7329863,"data_compressed":93914,"data_compressed_allocated":491520,"data_compressed_original":7030366,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509452,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303076,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068764,"statfs":{"total":96636764160,"available":96326414336,"internally_reserved":0,"allocated":1601536,"data_stored":13657406,"data_compressed":85801,"data_compressed_allocated":892928,"data_compressed_original":13326523,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306840,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065000,"statfs":{"total":96636764160,"available":96322560000,"internally_reserved":0,"allocated":1196032,"data_stored":5081013,"data_compressed":50623,"data_compressed_allocated":512000,"data_compressed_original":4739072,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302164,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069676,"statfs":{"total":96636764160,"available":96327348224,"internally_reserved":0,"allocated":659456,"data_stored":876915,"data_compressed":19116,"data_compressed_allocated":45056,"data_compressed_original":565248,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302668,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069172,"statfs":{"total":96636764160,"available":96326832128,"internally_reserved":0,"allocated":1171456,"data_stored":7361852,"data_compressed":94950,"data_compressed_allocated":491520,"data_compressed_original":7022174,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302560,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069280,"statfs":{"total":96636764160,"available":96326942720,"internally_reserved":0,"allocated":1069056,"data_stored":3396998,"data_compressed":76165,"data_compressed_allocated":405504,"data_compressed_original":3088384,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]}]},{"osd":1,"up_from":0,"seq":60129542156,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":6531105,"data_compressed":85612,"data_compressed_allocated":454656,"data_compressed_original":6448734,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":55525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":636536,"data_compressed":19116,"data_compressed_allocated":45056,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":4846792,"data_compressed":50623,"data_compressed_allocated":512000,"data_compressed_original":4739072,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":5785494,"data_compressed":51787,"data_compressed_allocated":417792,"data_compressed_original":5716867,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":598016,"data_stored":7089465,"data_compressed":93914,"data_compressed_allocated":491520,"data_compressed_original":7030366,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":323584,"data_stored":2524590,"data_compressed":29462,"data_compressed_allocated":221184,"data_compressed_original":2485123,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":2563015,"data_compressed":66568,"data_compressed_allocated":364544,"data_compressed_original":2457600,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T12:59:32.394 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T12:59:32.395 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:32.379+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:32.487 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:32.479+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:32.537 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:32.537 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:32.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:32.561 INFO:teuthology.orchestra.run.smithi042.stdout:1117 punch from 0x40af61d to 0x40b2d31, (0x3714 bytes) 2024-09-16T12:59:32.569 INFO:teuthology.orchestra.run.smithi042.stdout:1118 writesame 0xd5d9850 thru 0xd5dde54 data_size 0x19(0x4605 bytes) 2024-09-16T12:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout:1119 write 0x66e4aab thru 0x66f3c07 (0xf15d bytes) 2024-09-16T12:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout:1121 read 0xacfb0bd thru 0xacfde5d (0x2da1 bytes) 2024-09-16T12:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout:1122 punch from 0x4cce907 to 0x4cdb4b6, (0xcbaf bytes) 2024-09-16T12:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout:1123 read 0x81b9ef9 thru 0x81ba881 (0x989 bytes) 2024-09-16T12:59:32.589 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:32.583+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout:1124 trunc from 0xe8c8404 to 0x66e8f81 2024-09-16T12:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout:1125 write 0x2abcc60 thru 0x2abf065 (0x2406 bytes) 2024-09-16T12:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout:1126 read 0x476ddd5 thru 0x476fb86 (0x1db2 bytes) 2024-09-16T12:59:32.649 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:32.649 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:32.649 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:32.649 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:32.649 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:32.662 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:32.668 INFO:teuthology.orchestra.run.smithi042.stdout:1127 write 0xed4f244 thru 0xed59c6f (0xaa2c bytes) 2024-09-16T12:59:32.675 INFO:teuthology.orchestra.run.smithi042.stdout:1128 punch from 0x1a941f4 to 0x1aa33ec, (0xf1f8 bytes) 2024-09-16T12:59:32.675 INFO:teuthology.orchestra.run.smithi042.stdout:1129 punch from 0xc92617f to 0xc92be3a, (0x5cbb bytes) 2024-09-16T12:59:32.675 INFO:teuthology.orchestra.run.smithi042.stdout:1130 read 0xb949cae thru 0xb94d5db (0x392e bytes) 2024-09-16T12:59:32.675 INFO:teuthology.orchestra.run.smithi042.stdout:1131 read 0xa2cf214 thru 0xa2cfcb4 (0xaa1 bytes) 2024-09-16T12:59:32.675 INFO:teuthology.orchestra.run.smithi042.stdout:1133 write 0x6a87439 thru 0x6a8ca69 (0x5631 bytes) 2024-09-16T12:59:32.678 INFO:teuthology.orchestra.run.smithi042.stdout:1134 read 0x6bd2273 thru 0x6bdb0db (0x8e69 bytes) 2024-09-16T12:59:32.689 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:32.683+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:32.691 INFO:teuthology.orchestra.run.smithi042.stdout:1135 trunc from 0xed59c70 to 0x78ac21c 2024-09-16T12:59:32.711 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:32.711 INFO:teuthology.orchestra.run.smithi042.stdout:1137 punch from 0x5db53dd to 0x5dc0bfb, (0xb81e bytes) 2024-09-16T12:59:32.711 INFO:teuthology.orchestra.run.smithi042.stdout:1138 write 0x2ae0456 thru 0x2aeace2 (0xa88d bytes) 2024-09-16T12:59:32.715 INFO:teuthology.orchestra.run.smithi042.stdout:1140 read 0x68355a6 thru 0x6837fcd (0x2a28 bytes) 2024-09-16T12:59:32.715 INFO:teuthology.orchestra.run.smithi042.stdout:1141 read 0x5cee55b thru 0x5cef48f (0xf35 bytes) 2024-09-16T12:59:32.716 INFO:teuthology.orchestra.run.smithi042.stdout:1142 writesame 0xadaaa6 thru 0xadc156 data_size 0x1(0x16b1 bytes) 2024-09-16T12:59:32.739 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:59:32.740 INFO:teuthology.orchestra.run.smithi042.stdout:1143 writesame 0x7f091d2 thru 0x7f163ca data_size 0x7(0xd1f9 bytes) 2024-09-16T12:59:32.741 INFO:teuthology.orchestra.run.smithi042.stdout:1144 trunc from 0x7f163cb to 0x6eeca02 2024-09-16T12:59:32.743 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T12:59:32.755 INFO:teuthology.orchestra.run.smithi042.stdout:1146 trunc from 0x6eeca02 to 0x5e2d40c 2024-09-16T12:59:32.757 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-16T12:59:32.244364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":143850633,"num_objects":131,"num_object_clones":25,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":5,"num_read":11112,"num_read_kb":30592,"num_write":2033,"num_write_kb":55681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":55544,"num_keys_recovered":19,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1590,"ondisk_log_size":1590,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2122328,"kb_used_data":7552,"kb_used_omap":11,"kb_used_meta":2114676,"kb_avail":658480552,"statfs":{"total":676457349120,"available":674284085248,"internally_reserved":0,"allocated":7733248,"data_stored":40488504,"data_compressed":450031,"data_compressed_allocated":3059712,"data_compressed_original":38256890,"omap_allocated":11928,"internal_metadata":2165428584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39130300,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":3698,"num_read_kb":11958,"num_write":639,"num_write_kb":36792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001115"},"pg_stats":[{"pgid":"4.1f","version":"41'14","reported_seq":176,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.907151+0000","last_change":"2024-09-16T12:59:13.789786+0000","last_active":"2024-09-16T12:59:28.907151+0000","last_peered":"2024-09-16T12:59:28.907151+0000","last_clean":"2024-09-16T12:59:28.907151+0000","last_became_active":"2024-09-16T12:59:11.766798+0000","last_became_peered":"2024-09-16T12:59:11.766798+0000","last_unstale":"2024-09-16T12:59:28.907151+0000","last_undegraded":"2024-09-16T12:59:28.907151+0000","last_fullsized":"2024-09-16T12:59:28.907151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:22.588888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":1386038,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":142,"num_read_kb":1321,"num_write":27,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"47'61","reported_seq":128,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:29.739657+0000","last_change":"2024-09-16T12:59:14.030480+0000","last_active":"2024-09-16T12:59:29.739657+0000","last_peered":"2024-09-16T12:59:29.739657+0000","last_clean":"2024-09-16T12:59:29.739657+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T12:59:29.739657+0000","last_undegraded":"2024-09-16T12:59:29.739657+0000","last_fullsized":"2024-09-16T12:59:29.739657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049602200000000004,"stat_sum":{"num_bytes":12039604,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":81,"num_read_kb":787,"num_write":72,"num_write_kb":1212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"38'6","reported_seq":51,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.891885+0000","last_change":"2024-09-16T12:59:20.858214+0000","last_active":"2024-09-16T12:59:28.891885+0000","last_peered":"2024-09-16T12:59:28.891885+0000","last_clean":"2024-09-16T12:59:28.891885+0000","last_became_active":"2024-09-16T12:59:20.837194+0000","last_became_peered":"2024-09-16T12:59:20.837194+0000","last_unstale":"2024-09-16T12:59:28.891885+0000","last_undegraded":"2024-09-16T12:59:28.891885+0000","last_fullsized":"2024-09-16T12:59:28.891885+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:59.019644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012332339999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"45'47","reported_seq":147,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.883759+0000","last_change":"2024-09-16T12:59:20.868824+0000","last_active":"2024-09-16T12:59:28.883759+0000","last_peered":"2024-09-16T12:59:28.883759+0000","last_clean":"2024-09-16T12:59:28.883759+0000","last_became_active":"2024-09-16T12:59:20.837393+0000","last_became_peered":"2024-09-16T12:59:20.837393+0000","last_unstale":"2024-09-16T12:59:28.883759+0000","last_undegraded":"2024-09-16T12:59:28.883759+0000","last_fullsized":"2024-09-16T12:59:28.883759+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:51:34.761291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015138090000000001,"stat_sum":{"num_bytes":1059932,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":127,"num_read_kb":198,"num_write":55,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"41'9","reported_seq":42,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.885826+0000","last_change":"2024-09-16T12:59:13.779899+0000","last_active":"2024-09-16T12:59:28.885826+0000","last_peered":"2024-09-16T12:59:28.885826+0000","last_clean":"2024-09-16T12:59:28.885826+0000","last_became_active":"2024-09-16T12:59:11.759998+0000","last_became_peered":"2024-09-16T12:59:11.759998+0000","last_unstale":"2024-09-16T12:59:28.885826+0000","last_undegraded":"2024-09-16T12:59:28.885826+0000","last_fullsized":"2024-09-16T12:59:28.885826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:49:22.613940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002573915,"stat_sum":{"num_bytes":1867776,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":49,"num_write":10,"num_write_kb":1723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"44'46","reported_seq":559,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.221652+0000","last_change":"2024-09-16T12:59:13.779650+0000","last_active":"2024-09-16T12:59:30.221652+0000","last_peered":"2024-09-16T12:59:30.221652+0000","last_clean":"2024-09-16T12:59:30.221652+0000","last_became_active":"2024-09-16T12:59:11.759751+0000","last_became_peered":"2024-09-16T12:59:11.759751+0000","last_unstale":"2024-09-16T12:59:30.221652+0000","last_undegraded":"2024-09-16T12:59:30.221652+0000","last_fullsized":"2024-09-16T12:59:30.221652+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:12.049622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002565964,"stat_sum":{"num_bytes":32014824,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":512,"num_read_kb":4094,"num_write":45,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"41'15","reported_seq":49,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.891606+0000","last_change":"2024-09-16T12:59:13.770391+0000","last_active":"2024-09-16T12:59:28.891606+0000","last_peered":"2024-09-16T12:59:28.891606+0000","last_clean":"2024-09-16T12:59:28.891606+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T12:59:28.891606+0000","last_undegraded":"2024-09-16T12:59:28.891606+0000","last_fullsized":"2024-09-16T12:59:28.891606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013782849999999999,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":60,"num_write":20,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"47'67","reported_seq":228,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:29.541482+0000","last_change":"2024-09-16T12:59:13.777900+0000","last_active":"2024-09-16T12:59:29.541482+0000","last_peered":"2024-09-16T12:59:29.541482+0000","last_clean":"2024-09-16T12:59:29.541482+0000","last_became_active":"2024-09-16T12:59:11.767308+0000","last_became_peered":"2024-09-16T12:59:11.767308+0000","last_unstale":"2024-09-16T12:59:29.541482+0000","last_undegraded":"2024-09-16T12:59:29.541482+0000","last_fullsized":"2024-09-16T12:59:29.541482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:40.593369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093274999999999999,"stat_sum":{"num_bytes":5219078,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":404,"num_read_kb":1229,"num_write":110,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"45'17","reported_seq":175,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.892114+0000","last_change":"2024-09-16T12:59:13.770695+0000","last_active":"2024-09-16T12:59:28.892114+0000","last_peered":"2024-09-16T12:59:28.892114+0000","last_clean":"2024-09-16T12:59:28.892114+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T12:59:28.892114+0000","last_undegraded":"2024-09-16T12:59:28.892114+0000","last_fullsized":"2024-09-16T12:59:28.892114+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001481132,"stat_sum":{"num_bytes":5289573,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":133,"num_read_kb":507,"num_write":18,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"44'53","reported_seq":458,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.882816+0000","last_change":"2024-09-16T12:59:14.030762+0000","last_active":"2024-09-16T12:59:28.882816+0000","last_peered":"2024-09-16T12:59:28.882816+0000","last_clean":"2024-09-16T12:59:28.882816+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T12:59:28.882816+0000","last_undegraded":"2024-09-16T12:59:28.882816+0000","last_fullsized":"2024-09-16T12:59:28.882816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685533,"stat_sum":{"num_bytes":11165696,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":883,"num_read_kb":1173,"num_write":49,"num_write_kb":476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"48'90","reported_seq":335,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.165122+0000","last_change":"2024-09-16T12:59:13.796422+0000","last_active":"2024-09-16T12:59:30.165122+0000","last_peered":"2024-09-16T12:59:30.165122+0000","last_clean":"2024-09-16T12:59:30.165122+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T12:59:30.165122+0000","last_undegraded":"2024-09-16T12:59:30.165122+0000","last_fullsized":"2024-09-16T12:59:30.165122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055065799999999999,"stat_sum":{"num_bytes":12615862,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":569,"num_read_kb":1237,"num_write":108,"num_write_kb":695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"45'46","reported_seq":322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.891845+0000","last_change":"2024-09-16T12:59:20.883767+0000","last_active":"2024-09-16T12:59:28.891845+0000","last_peered":"2024-09-16T12:59:28.891845+0000","last_clean":"2024-09-16T12:59:28.891845+0000","last_became_active":"2024-09-16T12:59:20.842644+0000","last_became_peered":"2024-09-16T12:59:20.842644+0000","last_unstale":"2024-09-16T12:59:28.891845+0000","last_undegraded":"2024-09-16T12:59:28.891845+0000","last_fullsized":"2024-09-16T12:59:28.891845+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:18.381117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013482349999999999,"stat_sum":{"num_bytes":4562960,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":676,"num_read_kb":2189,"num_write":90,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"48'98","reported_seq":298,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.078097+0000","last_change":"2024-09-16T12:59:13.780086+0000","last_active":"2024-09-16T12:59:30.078097+0000","last_peered":"2024-09-16T12:59:30.078097+0000","last_clean":"2024-09-16T12:59:30.078097+0000","last_became_active":"2024-09-16T12:59:11.770883+0000","last_became_peered":"2024-09-16T12:59:11.770883+0000","last_unstale":"2024-09-16T12:59:30.078097+0000","last_undegraded":"2024-09-16T12:59:30.078097+0000","last_fullsized":"2024-09-16T12:59:30.078097+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:22.020502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024518000000000001,"stat_sum":{"num_bytes":17123005,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":471,"num_read_kb":803,"num_write":133,"num_write_kb":33479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"48'24","reported_seq":200,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:29.911060+0000","last_change":"2024-09-16T12:59:20.866963+0000","last_active":"2024-09-16T12:59:29.911060+0000","last_peered":"2024-09-16T12:59:29.911060+0000","last_clean":"2024-09-16T12:59:29.911060+0000","last_became_active":"2024-09-16T12:59:20.844338+0000","last_became_peered":"2024-09-16T12:59:20.844338+0000","last_unstale":"2024-09-16T12:59:29.911060+0000","last_undegraded":"2024-09-16T12:59:29.911060+0000","last_fullsized":"2024-09-16T12:59:29.911060+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:17.657544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061900799999999999,"stat_sum":{"num_bytes":559892,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":135,"num_read_kb":259,"num_write":23,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"43'10","reported_seq":46,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.888571+0000","last_change":"2024-09-16T12:59:13.770311+0000","last_active":"2024-09-16T12:59:28.888571+0000","last_peered":"2024-09-16T12:59:28.888571+0000","last_clean":"2024-09-16T12:59:28.888571+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T12:59:28.888571+0000","last_undegraded":"2024-09-16T12:59:28.888571+0000","last_fullsized":"2024-09-16T12:59:28.888571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413508,"stat_sum":{"num_bytes":376877,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":455,"num_write":9,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":37,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.888437+0000","last_change":"2024-09-16T12:59:20.835494+0000","last_active":"2024-09-16T12:59:28.888437+0000","last_peered":"2024-09-16T12:59:28.888437+0000","last_clean":"2024-09-16T12:59:28.888437+0000","last_became_active":"2024-09-16T12:59:20.835160+0000","last_became_peered":"2024-09-16T12:59:20.835160+0000","last_unstale":"2024-09-16T12:59:28.888437+0000","last_undegraded":"2024-09-16T12:59:28.888437+0000","last_fullsized":"2024-09-16T12:59:28.888437+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:42:23.395626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047952,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.907284+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T12:59:28.907284+0000","last_peered":"2024-09-16T12:59:28.907284+0000","last_clean":"2024-09-16T12:59:28.907284+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T12:59:28.907284+0000","last_undegraded":"2024-09-16T12:59:28.907284+0000","last_fullsized":"2024-09-16T12:59:28.907284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"48'35","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.001833+0000","last_change":"2024-09-16T12:59:13.777645+0000","last_active":"2024-09-16T12:59:30.001833+0000","last_peered":"2024-09-16T12:59:30.001833+0000","last_clean":"2024-09-16T12:59:30.001833+0000","last_became_active":"2024-09-16T12:59:11.770325+0000","last_became_peered":"2024-09-16T12:59:11.770325+0000","last_unstale":"2024-09-16T12:59:30.001833+0000","last_undegraded":"2024-09-16T12:59:30.001833+0000","last_fullsized":"2024-09-16T12:59:30.001833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:06:31.319709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069048899999999997,"stat_sum":{"num_bytes":36,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":79,"num_write":34,"num_write_kb":689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.907049+0000","last_change":"2024-09-16T12:59:20.842195+0000","last_active":"2024-09-16T12:59:28.907049+0000","last_peered":"2024-09-16T12:59:28.907049+0000","last_clean":"2024-09-16T12:59:28.907049+0000","last_became_active":"2024-09-16T12:59:20.841956+0000","last_became_peered":"2024-09-16T12:59:20.841956+0000","last_unstale":"2024-09-16T12:59:28.907049+0000","last_undegraded":"2024-09-16T12:59:28.907049+0000","last_fullsized":"2024-09-16T12:59:28.907049+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:53:07.991519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"48'128","reported_seq":698,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.266386+0000","last_change":"2024-09-16T12:59:13.769781+0000","last_active":"2024-09-16T12:59:30.266386+0000","last_peered":"2024-09-16T12:59:30.266386+0000","last_clean":"2024-09-16T12:59:30.266386+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T12:59:30.266386+0000","last_undegraded":"2024-09-16T12:59:30.266386+0000","last_fullsized":"2024-09-16T12:59:30.266386+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010481990000000001,"stat_sum":{"num_bytes":137,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1612,"num_read_kb":1394,"num_write":147,"num_write_kb":364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T12:59:27.896946+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T12:59:27.896946+0000","last_peered":"2024-09-16T12:59:27.896946+0000","last_clean":"2024-09-16T12:59:27.896946+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T12:59:27.896946+0000","last_undegraded":"2024-09-16T12:59:27.896946+0000","last_fullsized":"2024-09-16T12:59:27.896946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"42'19","reported_seq":182,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.883017+0000","last_change":"2024-09-16T12:59:14.030767+0000","last_active":"2024-09-16T12:59:28.883017+0000","last_peered":"2024-09-16T12:59:28.883017+0000","last_clean":"2024-09-16T12:59:28.883017+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T12:59:28.883017+0000","last_undegraded":"2024-09-16T12:59:28.883017+0000","last_fullsized":"2024-09-16T12:59:28.883017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000670212,"stat_sum":{"num_bytes":1807886,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":139,"num_read_kb":1266,"num_write":24,"num_write_kb":623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.882713+0000","last_change":"2024-09-16T12:59:20.840516+0000","last_active":"2024-09-16T12:59:28.882713+0000","last_peered":"2024-09-16T12:59:28.882713+0000","last_clean":"2024-09-16T12:59:28.882713+0000","last_became_active":"2024-09-16T12:59:20.840106+0000","last_became_peered":"2024-09-16T12:59:20.840106+0000","last_unstale":"2024-09-16T12:59:28.882713+0000","last_undegraded":"2024-09-16T12:59:28.882713+0000","last_fullsized":"2024-09-16T12:59:28.882713+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:27.770889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"42'32","reported_seq":192,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.896119+0000","last_change":"2024-09-16T12:59:20.878776+0000","last_active":"2024-09-16T12:59:28.896119+0000","last_peered":"2024-09-16T12:59:28.896119+0000","last_clean":"2024-09-16T12:59:28.896119+0000","last_became_active":"2024-09-16T12:59:20.845288+0000","last_became_peered":"2024-09-16T12:59:20.845288+0000","last_unstale":"2024-09-16T12:59:28.896119+0000","last_undegraded":"2024-09-16T12:59:28.896119+0000","last_fullsized":"2024-09-16T12:59:28.896119+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:11.492412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045226000000000001,"stat_sum":{"num_bytes":1327120,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":348,"num_read_kb":737,"num_write":27,"num_write_kb":337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8208,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.885925+0000","last_change":"2024-09-16T12:59:01.688552+0000","last_active":"2024-09-16T12:59:28.885925+0000","last_peered":"2024-09-16T12:59:28.885925+0000","last_clean":"2024-09-16T12:59:28.885925+0000","last_became_active":"2024-09-16T12:59:01.688068+0000","last_became_peered":"2024-09-16T12:59:01.688068+0000","last_unstale":"2024-09-16T12:59:28.885925+0000","last_undegraded":"2024-09-16T12:59:28.885925+0000","last_fullsized":"2024-09-16T12:59:28.885925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:01.298345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":54,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.896034+0000","last_change":"2024-09-16T12:59:20.882735+0000","last_active":"2024-09-16T12:59:28.896034+0000","last_peered":"2024-09-16T12:59:28.896034+0000","last_clean":"2024-09-16T12:59:28.896034+0000","last_became_active":"2024-09-16T12:59:20.845568+0000","last_became_peered":"2024-09-16T12:59:20.845568+0000","last_unstale":"2024-09-16T12:59:28.896034+0000","last_undegraded":"2024-09-16T12:59:28.896034+0000","last_fullsized":"2024-09-16T12:59:28.896034+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:18.023864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"41'18","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.883698+0000","last_change":"2024-09-16T12:59:20.849504+0000","last_active":"2024-09-16T12:59:28.883698+0000","last_peered":"2024-09-16T12:59:28.883698+0000","last_clean":"2024-09-16T12:59:28.883698+0000","last_became_active":"2024-09-16T12:59:20.832759+0000","last_became_peered":"2024-09-16T12:59:20.832759+0000","last_unstale":"2024-09-16T12:59:28.883698+0000","last_undegraded":"2024-09-16T12:59:28.883698+0000","last_fullsized":"2024-09-16T12:59:28.883698+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:23.812603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077242,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":37,"num_write":17,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.883323+0000","last_change":"2024-09-16T12:59:20.835231+0000","last_active":"2024-09-16T12:59:28.883323+0000","last_peered":"2024-09-16T12:59:28.883323+0000","last_clean":"2024-09-16T12:59:28.883323+0000","last_became_active":"2024-09-16T12:59:20.834767+0000","last_became_peered":"2024-09-16T12:59:20.834767+0000","last_unstale":"2024-09-16T12:59:28.883323+0000","last_undegraded":"2024-09-16T12:59:28.883323+0000","last_fullsized":"2024-09-16T12:59:28.883323+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:28.665721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.883486+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T12:59:28.883486+0000","last_peered":"2024-09-16T12:59:28.883486+0000","last_clean":"2024-09-16T12:59:28.883486+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T12:59:28.883486+0000","last_undegraded":"2024-09-16T12:59:28.883486+0000","last_fullsized":"2024-09-16T12:59:28.883486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"44'19","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.896073+0000","last_change":"2024-09-16T12:59:20.852417+0000","last_active":"2024-09-16T12:59:28.896073+0000","last_peered":"2024-09-16T12:59:28.896073+0000","last_clean":"2024-09-16T12:59:28.896073+0000","last_became_active":"2024-09-16T12:59:20.840157+0000","last_became_peered":"2024-09-16T12:59:20.840157+0000","last_unstale":"2024-09-16T12:59:28.896073+0000","last_undegraded":"2024-09-16T12:59:28.896073+0000","last_fullsized":"2024-09-16T12:59:28.896073+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:14.383652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064957100000000005,"stat_sum":{"num_bytes":10633216,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":31,"num_read_kb":4226,"num_write":18,"num_write_kb":4239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.882581+0000","last_change":"2024-09-16T12:59:01.686741+0000","last_active":"2024-09-16T12:59:28.882581+0000","last_peered":"2024-09-16T12:59:28.882581+0000","last_clean":"2024-09-16T12:59:28.882581+0000","last_became_active":"2024-09-16T12:59:01.686267+0000","last_became_peered":"2024-09-16T12:59:01.686267+0000","last_unstale":"2024-09-16T12:59:28.882581+0000","last_undegraded":"2024-09-16T12:59:28.882581+0000","last_fullsized":"2024-09-16T12:59:28.882581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:49.310418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T12:59:27.897055+0000","last_change":"2024-09-16T12:59:20.840160+0000","last_active":"2024-09-16T12:59:27.897055+0000","last_peered":"2024-09-16T12:59:27.897055+0000","last_clean":"2024-09-16T12:59:27.897055+0000","last_became_active":"2024-09-16T12:59:20.839853+0000","last_became_peered":"2024-09-16T12:59:20.839853+0000","last_unstale":"2024-09-16T12:59:27.897055+0000","last_undegraded":"2024-09-16T12:59:27.897055+0000","last_fullsized":"2024-09-16T12:59:27.897055+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:45.235735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"41'6","reported_seq":179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.882966+0000","last_change":"2024-09-16T12:59:20.845862+0000","last_active":"2024-09-16T12:59:28.882966+0000","last_peered":"2024-09-16T12:59:28.882966+0000","last_clean":"2024-09-16T12:59:28.882966+0000","last_became_active":"2024-09-16T12:59:20.839259+0000","last_became_peered":"2024-09-16T12:59:20.839259+0000","last_unstale":"2024-09-16T12:59:28.882966+0000","last_undegraded":"2024-09-16T12:59:28.882966+0000","last_fullsized":"2024-09-16T12:59:28.882966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:26:22.372402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094804399999999999,"stat_sum":{"num_bytes":1194197,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":1278,"num_write":5,"num_write_kb":255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":30933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.891643+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T12:59:28.891643+0000","last_peered":"2024-09-16T12:59:28.891643+0000","last_clean":"2024-09-16T12:59:28.891643+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T12:59:28.891643+0000","last_undegraded":"2024-09-16T12:59:28.891643+0000","last_fullsized":"2024-09-16T12:59:28.891643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:20.468036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":40,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.896040+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T12:59:28.896040+0000","last_peered":"2024-09-16T12:59:28.896040+0000","last_clean":"2024-09-16T12:59:28.896040+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T12:59:28.896040+0000","last_undegraded":"2024-09-16T12:59:28.896040+0000","last_fullsized":"2024-09-16T12:59:28.896040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":127,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T12:59:27.896607+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T12:59:27.896607+0000","last_peered":"2024-09-16T12:59:27.896607+0000","last_clean":"2024-09-16T12:59:27.896607+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T12:59:27.896607+0000","last_undegraded":"2024-09-16T12:59:27.896607+0000","last_fullsized":"2024-09-16T12:59:27.896607+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"45'16","reported_seq":187,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.906952+0000","last_change":"2024-09-16T12:59:13.789075+0000","last_active":"2024-09-16T12:59:28.906952+0000","last_peered":"2024-09-16T12:59:28.906952+0000","last_clean":"2024-09-16T12:59:28.906952+0000","last_became_active":"2024-09-16T12:59:11.766779+0000","last_became_peered":"2024-09-16T12:59:11.766779+0000","last_unstale":"2024-09-16T12:59:28.906952+0000","last_undegraded":"2024-09-16T12:59:28.906952+0000","last_fullsized":"2024-09-16T12:59:28.906952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:47.829432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046786199999999999,"stat_sum":{"num_bytes":1209022,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":158,"num_read_kb":344,"num_write":15,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"42'166","reported_seq":371,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.895958+0000","last_change":"2024-09-16T12:59:13.796421+0000","last_active":"2024-09-16T12:59:28.895958+0000","last_peered":"2024-09-16T12:59:28.895958+0000","last_clean":"2024-09-16T12:59:28.895958+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T12:59:28.895958+0000","last_undegraded":"2024-09-16T12:59:28.895958+0000","last_fullsized":"2024-09-16T12:59:28.895958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067313300000000002,"stat_sum":{"num_bytes":5512502,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":939,"num_read_kb":2071,"num_write":264,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"42'50","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.885567+0000","last_change":"2024-09-16T12:59:20.859488+0000","last_active":"2024-09-16T12:59:28.885567+0000","last_peered":"2024-09-16T12:59:28.885567+0000","last_clean":"2024-09-16T12:59:28.885567+0000","last_became_active":"2024-09-16T12:59:20.835273+0000","last_became_peered":"2024-09-16T12:59:20.835273+0000","last_unstale":"2024-09-16T12:59:28.885567+0000","last_undegraded":"2024-09-16T12:59:28.885567+0000","last_fullsized":"2024-09-16T12:59:28.885567+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:17:56.825719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011000280000000001,"stat_sum":{"num_bytes":5144612,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":188,"num_write":72,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"48'61","reported_seq":265,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.212468+0000","last_change":"2024-09-16T12:59:20.859192+0000","last_active":"2024-09-16T12:59:30.212468+0000","last_peered":"2024-09-16T12:59:30.212468+0000","last_clean":"2024-09-16T12:59:30.212468+0000","last_became_active":"2024-09-16T12:59:20.842251+0000","last_became_peered":"2024-09-16T12:59:20.842251+0000","last_unstale":"2024-09-16T12:59:30.212468+0000","last_undegraded":"2024-09-16T12:59:30.212468+0000","last_fullsized":"2024-09-16T12:59:30.212468+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:24.239482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078078700000000002,"stat_sum":{"num_bytes":5136384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":480,"num_read_kb":519,"num_write":56,"num_write_kb":527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":8192,"num_keys_recovered":2,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"45'15","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.882635+0000","last_change":"2024-09-16T12:59:20.864100+0000","last_active":"2024-09-16T12:59:28.882635+0000","last_peered":"2024-09-16T12:59:28.882635+0000","last_clean":"2024-09-16T12:59:28.882635+0000","last_became_active":"2024-09-16T12:59:20.842488+0000","last_became_peered":"2024-09-16T12:59:20.842488+0000","last_unstale":"2024-09-16T12:59:28.882635+0000","last_undegraded":"2024-09-16T12:59:28.882635+0000","last_fullsized":"2024-09-16T12:59:28.882635+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:22.250517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053245600000000005,"stat_sum":{"num_bytes":55,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":69,"num_write":20,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"39'32","reported_seq":288,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.885922+0000","last_change":"2024-09-16T12:59:13.780281+0000","last_active":"2024-09-16T12:59:28.885922+0000","last_peered":"2024-09-16T12:59:28.885922+0000","last_clean":"2024-09-16T12:59:28.885922+0000","last_became_active":"2024-09-16T12:59:11.764080+0000","last_became_peered":"2024-09-16T12:59:11.764080+0000","last_unstale":"2024-09-16T12:59:28.885922+0000","last_undegraded":"2024-09-16T12:59:28.885922+0000","last_fullsized":"2024-09-16T12:59:28.885922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:55:18.557326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020639640000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":688,"num_read_kb":600,"num_write":28,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"43'18","reported_seq":193,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.886039+0000","last_change":"2024-09-16T12:59:13.780488+0000","last_active":"2024-09-16T12:59:28.886039+0000","last_peered":"2024-09-16T12:59:28.886039+0000","last_clean":"2024-09-16T12:59:28.886039+0000","last_became_active":"2024-09-16T12:59:11.764699+0000","last_became_peered":"2024-09-16T12:59:11.764699+0000","last_unstale":"2024-09-16T12:59:28.886039+0000","last_undegraded":"2024-09-16T12:59:28.886039+0000","last_fullsized":"2024-09-16T12:59:28.886039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:12.151943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001758094,"stat_sum":{"num_bytes":1410023,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":601,"num_write":18,"num_write_kb":228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"48'287","reported_seq":689,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T12:59:30.258595+0000","last_change":"2024-09-16T12:59:13.780729+0000","last_active":"2024-09-16T12:59:30.258595+0000","last_peered":"2024-09-16T12:59:30.258595+0000","last_clean":"2024-09-16T12:59:30.258595+0000","last_became_active":"2024-09-16T12:59:11.770833+0000","last_became_peered":"2024-09-16T12:59:11.770833+0000","last_unstale":"2024-09-16T12:59:30.258595+0000","last_undegraded":"2024-09-16T12:59:30.258595+0000","last_fullsized":"2024-09-16T12:59:30.258595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:56.784122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637831,"stat_sum":{"num_bytes":2056227,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1929,"num_read_kb":2703,"num_write":399,"num_write_kb":1248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"41'10","reported_seq":52,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T12:59:28.891872+0000","last_change":"2024-09-16T12:59:20.880782+0000","last_active":"2024-09-16T12:59:28.891872+0000","last_peered":"2024-09-16T12:59:28.891872+0000","last_clean":"2024-09-16T12:59:28.891872+0000","last_became_active":"2024-09-16T12:59:20.839661+0000","last_became_peered":"2024-09-16T12:59:20.839661+0000","last_unstale":"2024-09-16T12:59:28.891872+0000","last_undegraded":"2024-09-16T12:59:28.891872+0000","last_fullsized":"2024-09-16T12:59:28.891872+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:11.767806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053943399999999999,"stat_sum":{"num_bytes":647168,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":64,"num_write":11,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":143260246,"num_objects":127,"num_object_clones":25,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":5,"num_read":11046,"num_read_kb":30537,"num_write":1930,"num_write_kb":54051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":55525,"num_keys_recovered":19,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3440640,"data_stored":30032522,"data_compressed":397082,"data_compressed_allocated":2506752,"data_compressed_original":29443010,"omap_allocated":0,"internal_metadata":0},"log_size":1515,"ondisk_log_size":1515,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302384,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069456,"statfs":{"total":96636764160,"available":96327122944,"internally_reserved":0,"allocated":892928,"data_stored":2784457,"data_compressed":29462,"data_compressed_allocated":221184,"data_compressed_original":2485123,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302636,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069204,"statfs":{"total":96636764160,"available":96326864896,"internally_reserved":0,"allocated":1142784,"data_stored":7329863,"data_compressed":93914,"data_compressed_allocated":491520,"data_compressed_original":7030366,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509452,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303076,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068764,"statfs":{"total":96636764160,"available":96326414336,"internally_reserved":0,"allocated":1601536,"data_stored":13657406,"data_compressed":85801,"data_compressed_allocated":892928,"data_compressed_original":13326523,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306840,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065000,"statfs":{"total":96636764160,"available":96322560000,"internally_reserved":0,"allocated":1196032,"data_stored":5081013,"data_compressed":50623,"data_compressed_allocated":512000,"data_compressed_original":4739072,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302164,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069676,"statfs":{"total":96636764160,"available":96327348224,"internally_reserved":0,"allocated":659456,"data_stored":876915,"data_compressed":19116,"data_compressed_allocated":45056,"data_compressed_original":565248,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302668,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069172,"statfs":{"total":96636764160,"available":96326832128,"internally_reserved":0,"allocated":1171456,"data_stored":7361852,"data_compressed":94950,"data_compressed_allocated":491520,"data_compressed_original":7022174,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302560,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069280,"statfs":{"total":96636764160,"available":96326942720,"internally_reserved":0,"allocated":1069056,"data_stored":3396998,"data_compressed":76165,"data_compressed_allocated":405504,"data_compressed_original":3088384,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]}]},{"osd":1,"up_from":0,"seq":60129542156,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":6531105,"data_compressed":85612,"data_compressed_allocated":454656,"data_compressed_original":6448734,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":55525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":636536,"data_compressed":19116,"data_compressed_allocated":45056,"data_compressed_original":565248,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":4846792,"data_compressed":50623,"data_compressed_allocated":512000,"data_compressed_original":4739072,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":5785494,"data_compressed":51787,"data_compressed_allocated":417792,"data_compressed_original":5716867,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":598016,"data_stored":7089465,"data_compressed":93914,"data_compressed_allocated":491520,"data_compressed_original":7030366,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":323584,"data_stored":2524590,"data_compressed":29462,"data_compressed_allocated":221184,"data_compressed_original":2485123,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":2563015,"data_compressed":66568,"data_compressed_allocated":364544,"data_compressed_original":2457600,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T12:59:32.759 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T12:59:32.759 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T12:59:32.759 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T12:59:32.759 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-09-16T12:59:32.759 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T12:59:32.768 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:313 4.1e 4:7abe541a:::rbd_object_map.11b524d5182b.0000000000000004:head [writefull 0~35] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.958957024, 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053862449999999996, 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 313 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804330+0000", 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3309999999999998e-06 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804341+0000", 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1284000000000001e-05 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804367+0000", 2024-09-16T12:59:32.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6132999999999999e-05 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804380+0000", 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2686e-05 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804463+0000", 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3187000000000002e-05 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:32.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804472+0000", 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.149e-06 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.806749+0000", 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022770669999999998 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.806912+0000", 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016236500000000001 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.807071+0000", 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000158956 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.808001+0000", 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093026199999999995 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809575+0000", 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015741609999999999 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809654+0000", 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9028000000000003e-05 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809714+0000", 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9636000000000003e-05 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:486 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:32.775 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.656973158, 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077714180000000004, 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 486 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106314+0000", 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8329999999999998e-06 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106329+0000", 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4231e-05 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106369+0000", 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0074999999999998e-05 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106383+0000", 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4781e-05 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106471+0000", 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7472999999999994e-05 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106667+0000", 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000196503 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.107835+0000", 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011676670000000001 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.113820+0000", 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0059854069999999999 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.113900+0000", 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0071e-05 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.114010+0000", 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109399 2024-09-16T12:59:32.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.114083+0000", 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2978000000000005e-05 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:636 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.291085572, 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027791489999999999, 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 636 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472202+0000", 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.01e-06 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472218+0000", 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5724000000000001e-05 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472261+0000", 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2858999999999997e-05 2024-09-16T12:59:32.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472277+0000", 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6297999999999999e-05 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472397+0000", 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012026199999999999 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472563+0000", 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016614900000000001 2024-09-16T12:59:32.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.480681+0000", 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081177869999999996 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499671+0000", 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018990289 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499756+0000", 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4511000000000005e-05 2024-09-16T12:59:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499858+0000", 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102495 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499991+0000", 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132106 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.786 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:846 4.b 4:d203d4b6:::rbd_data.11b59d02cc38.0000000000000013:head [read 655360~35183] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.967702+0000", 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.795582358000001, 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038915310000000002, 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 846 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967702+0000", 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967702+0000", 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967705+0000", 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1659999999999998e-06 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967719+0000", 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-05 2024-09-16T12:59:32.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967753+0000", 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4866999999999997e-05 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967766+0000", 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.272e-05 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967870+0000", 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010409800000000001 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967880+0000", 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9559999999999992e-06 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.971326+0000", 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034458790000000002 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.971432+0000", 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010626800000000001 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.971594+0000", 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016137700000000001 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:989 4.b 4:d203d4b6:::rbd_data.11b59d02cc38.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.304473+0000", 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.458811982, 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038852840000000001, 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 989 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304473+0000", 2024-09-16T12:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304473+0000", 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304475+0000", 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7539999999999998e-06 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304484+0000", 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8380000000000008e-06 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304510+0000", 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5820999999999999e-05 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304520+0000", 2024-09-16T12:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7010000000000008e-06 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304641+0000", 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120951 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304741+0000", 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010043300000000001 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304869+0000", 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012801699999999999 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.306507+0000", 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001637552 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308257+0000", 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017501179999999999 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308313+0000", 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6060999999999997e-05 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308358+0000", 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5037999999999998e-05 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1081 4.3 4:c0127367:::rbd_data.11b59d02cc38.00000000000000ac:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.443370+0000", 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.319915001, 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043702059999999997, 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443370+0000", 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443370+0000", 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443372+0000", 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1459999999999999e-06 2024-09-16T12:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443394+0000", 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2410000000000001e-05 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443422+0000", 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7464e-05 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443452+0000", 2024-09-16T12:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0241000000000001e-05 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443558+0000", 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644700000000001 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.443837+0000", 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027904500000000001 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.444488+0000", 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065053099999999998 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.447187+0000", 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026991390000000001 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.447603+0000", 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041617100000000001 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.447679+0000", 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5927000000000007e-05 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.447740+0000", 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0684999999999999e-05 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3577 4.1e 4:7a8a5a14:::rbd_data.11b5c378985b.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.66263912, 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027798529999999998, 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3577 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100649+0000", 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1769999999999998e-06 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100659+0000", 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0407e-05 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100689+0000", 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0290999999999998e-05 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100701+0000", 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1960999999999999e-05 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100800+0000", 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8350000000000005e-05 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.101027+0000", 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000227736 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.111830+0000", 2024-09-16T12:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010802735000000001 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128285+0000", 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016454389999999999 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128338+0000", 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3112000000000002e-05 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128398+0000", 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0136999999999997e-05 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128444+0000", 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6233999999999998e-05 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:3577 4.1e e41/31 ondisk, result = 0)", 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.643473084, 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086403560000000001, 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119813+0000", 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4950000000000001e-06 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119822+0000", 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4210000000000001e-06 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119840+0000", 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7824000000000001e-05 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119848+0000", 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9750000000000007e-06 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128321+0000", 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084731100000000007 2024-09-16T12:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128330+0000", 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619999999999998e-06 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128452+0000", 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012226899999999999 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3672 4.16 4:6b60bac8:::rbd_header.11b5c378985b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.839314+0000", 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9239710779999992, 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055860959999999996, 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3672 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839314+0000", 2024-09-16T12:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839314+0000", 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839316+0000", 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2460000000000002e-06 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839326+0000", 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0662000000000001e-05 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839352+0000", 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5086999999999999e-05 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839360+0000", 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9029999999999993e-06 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839453+0000", 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2719000000000005e-05 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839465+0000", 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1798e-05 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.843224+0000", 2024-09-16T12:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037588460000000001 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.843301+0000", 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7619000000000003e-05 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.843630+0000", 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032843999999999999 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844264+0000", 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063367499999999999 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844809+0000", 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00054593999999999997 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844844+0000", 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4603000000000001e-05 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844900+0000", 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5557999999999999e-05 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3894 4.1e 4:7bf418e8:::rbd_data.11b53140b009.0000000000000054:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.135227+0000", 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.6280579660000001, 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049140659999999999, 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3894 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135227+0000", 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135227+0000", 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135228+0000", 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.15e-06 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135233+0000", 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0640000000000001e-06 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135250+0000", 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7334e-05 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135261+0000", 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0407e-05 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.137529+0000", 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022684039999999999 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.138104+0000", 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057520100000000001 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140141+0000", 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002036506 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3929 4.1e 4:7922d7a2:::rbd_data.11b53140b009.0000000000000077:head [sparse-read 0~229376] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.136167+0000", 2024-09-16T12:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.6271177580000007, 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042116530000000001, 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3929 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136167+0000", 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136167+0000", 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136168+0000", 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0780000000000001e-06 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136173+0000", 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8860000000000003e-06 2024-09-16T12:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136200+0000", 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7650000000000001e-05 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136207+0000", 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7399999999999998e-06 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140172+0000", 2024-09-16T12:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039647659999999998 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140245+0000", 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3323000000000003e-05 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140378+0000", 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013321000000000001 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3932 4.6 4:61b26f91:::rbd_data.11b53140b009.000000000000007a:head [sparse-read 0~229376] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.6270260939999996, 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046959480000000001, 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3932 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.37e-07 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136261+0000", 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2000000000000001e-06 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136968+0000", 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070683599999999997 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136983+0000", 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4990999999999999e-05 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140450+0000", 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003467131 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140634+0000", 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018384400000000001 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140954+0000", 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032030900000000001 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5066 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.341217007, 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019002768999999999, 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5066 2024-09-16T12:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422070+0000", 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-06 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422082+0000", 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2224e-05 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422120+0000", 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7662e-05 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422133+0000", 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3336000000000001e-05 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422296+0000", 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016319899999999999 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422539+0000", 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000243281 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.427723+0000", 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051837580000000001 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.440864+0000", 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013140446 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.440892+0000", 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8628000000000001e-05 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.441002+0000", 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010946999999999999 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.441070+0000", 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8595000000000003e-05 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5133 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write 20651826~62206] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.572603+0000", 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1906817209999998, 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039609670000000001, 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5133 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.572603+0000", 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.572603+0000", 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.572605+0000", 2024-09-16T12:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0269999999999998e-06 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.572680+0000", 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5152000000000001e-05 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.572714+0000", 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3915000000000003e-05 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.572726+0000", 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2003e-05 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.572832+0000", 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106424 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.572979+0000", 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000146434 2024-09-16T12:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.573206+0000", 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000226927 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.574934+0000", 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00172802 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.576378+0000", 2024-09-16T12:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001444543 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.576459+0000", 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0941999999999999e-05 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.576564+0000", 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010458 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5229 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9250627930000004, 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049016400000000002, 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5229 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838225+0000", 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0340000000000001e-06 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838239+0000", 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4004e-05 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838280+0000", 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0738000000000002e-05 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838294+0000", 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.482e-05 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838415+0000", 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012084 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838570+0000", 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000155162 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.850428+0000", 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011857826 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887135+0000", 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036706938000000001 2024-09-16T12:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887149+0000", 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3421000000000001e-05 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887202+0000", 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3427000000000002e-05 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887238+0000", 2024-09-16T12:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6189999999999997e-05 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:5229 4.1e e45/31 ondisk, result = 0)", 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.874885+0000", 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.8883993459999999, 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012252984, 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874885+0000", 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874885+0000", 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874887+0000", 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.874e-06 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874894+0000", 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4529999999999999e-06 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874917+0000", 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3125999999999998e-05 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874928+0000", 2024-09-16T12:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1139e-05 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887109+0000", 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012181290000000001 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887125+0000", 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5529e-05 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887138+0000", 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3573e-05 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5230 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.887669+0000", 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.8756159849999996, 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005832523, 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887669+0000", 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887669+0000", 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887671+0000", 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2010000000000002e-06 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887681+0000", 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0465000000000001e-05 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887701+0000", 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9709999999999999e-05 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887709+0000", 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6960000000000005e-06 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887778+0000", 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9363999999999995e-05 2024-09-16T12:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887784+0000", 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2129999999999996e-06 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.890175+0000", 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002390381 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.890275+0000", 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9994999999999994e-05 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.890554+0000", 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027887099999999998 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.892125+0000", 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015718539999999999 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.893435+0000", 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013094859999999999 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.893474+0000", 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8708999999999998e-05 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.893501+0000", 2024-09-16T12:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7577999999999999e-05 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5396 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5412416699999998, 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035598629999999999, 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5396 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222045+0000", 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9580000000000001e-06 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222055+0000", 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9310000000000003e-06 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222082+0000", 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6882000000000001e-05 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222092+0000", 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8910000000000007e-06 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222166+0000", 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4437999999999994e-05 2024-09-16T12:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222264+0000", 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7748e-05 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.228318+0000", 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060540109999999998 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257411+0000", 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029093622999999999 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257500+0000", 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8341e-05 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257580+0000", 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257642+0000", 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1073000000000003e-05 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:5396 4.1e e46/31 ondisk, result = 0)", 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.248841+0000", 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5144439119999999, 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088103839999999992, 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248841+0000", 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248841+0000", 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248842+0000", 2024-09-16T12:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.012e-06 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248848+0000", 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.947e-06 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248865+0000", 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7588000000000002e-05 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248873+0000", 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257476+0000", 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086037709999999996 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257489+0000", 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2334000000000001e-05 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257651+0000", 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016232799999999999 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6557 4.6 4:62ff1677:::rbd_object_map.11b53fd98b75.000000000000000b:head [writefull 0~36] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:31.923520+0000", 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.83976452400000001, 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053363129999999996, 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6557 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923520+0000", 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923520+0000", 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923523+0000", 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923538+0000", 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4865000000000001e-05 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923573+0000", 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5000999999999999e-05 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923587+0000", 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3505e-05 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923668+0000", 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260999999999993e-05 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923675+0000", 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0199999999999997e-06 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.926586+0000", 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029107730000000002 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.926827+0000", 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000240898 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.927056+0000", 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022953200000000001 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928356+0000", 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012995999999999999 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928668+0000", 2024-09-16T12:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031215300000000002 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928792+0000", 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000124191 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928856+0000", 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4193e-05 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:32.879 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:32.879 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:32.881 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:32.882 INFO:teuthology.orchestra.run.smithi042.stdout:1149 write 0xe967305 thru 0xe973a57 (0xc753 bytes) 2024-09-16T12:59:32.882 INFO:teuthology.orchestra.run.smithi042.stdout:1152 read 0x1687dec thru 0x168813d (0x352 bytes) 2024-09-16T12:59:32.882 INFO:teuthology.orchestra.run.smithi042.stdout:1153 punch from 0x7c3f65 to 0x7cecda, (0xad75 bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1155 read 0x7a05d96 thru 0x7a11de8 (0xc053 bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1157 punch from 0x696d787 to 0x697203e, (0x48b7 bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1159 punch from 0x4cff463 to 0x4d0c0c1, (0xcc5e bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1160 writesame 0x2b94197 thru 0x2b9aeec data_size 0x2d(0x6d56 bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1161 write 0x2aabbe6 thru 0x2ab138c (0x57a7 bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1162 punch from 0xe6a71e8 to 0xe6abf00, (0x4d18 bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1163 writesame 0x6bf2757 thru 0x6bf9416 data_size 0x28(0x6cc0 bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1164 write 0x3fd7f91 thru 0x3fd8be9 (0xc59 bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1165 read 0x82c8aa2 thru 0x82d2511 (0x9a70 bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1166 write 0x5df79c1 thru 0x5e033ee (0xba2e bytes) 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1167 trunc from 0xe973a58 to 0xa181a1d 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:1172 clone 9 order 22 su 524288 sc 5 2024-09-16T12:59:32.884 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:32.875+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:32.981 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:32.971+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:32.988 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:32.988 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:32.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:33.031 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone8 from 0xa181a1d (overlap 0x5e2d40c) to 0x1c40c6a 2024-09-16T12:59:33.082 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:33.079+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:33.105 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:33.105 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:33.112 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:33.176 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-16T12:59:33.183 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:33.179+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:33.209 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:346 4.2 4:4550e1e5:::rbd_id.image_client.0-clone1:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.352462005, 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003768212, 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 346 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850713+0000", 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6270000000000001e-06 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850722+0000", 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2169999999999994e-06 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850739+0000", 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7005000000000001e-05 2024-09-16T12:59:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850758+0000", 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9502999999999999e-05 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850826+0000", 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8032000000000002e-05 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850959+0000", 2024-09-16T12:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013249600000000001 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.851164+0000", 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020556 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.852079+0000", 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091475699999999998 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854331+0000", 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022523040000000001 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854400+0000", 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8356999999999996e-05 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854480+0000", 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0353999999999997e-05 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:543 4.7 4:e655f1b3:::rbd_data.11b59d02cc38.000000000000004c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 367827~25389] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.967855591999999, 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004583953, 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 543 2024-09-16T12:59:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235321+0000", 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.117e-06 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235354+0000", 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2962000000000001e-05 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235394+0000", 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9954999999999999e-05 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235409+0000", 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4756999999999999e-05 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235553+0000", 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000144284 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235809+0000", 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025601299999999998 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.236081+0000", 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000272093 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237888+0000", 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806298 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239756+0000", 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001867884 2024-09-16T12:59:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239842+0000", 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6581000000000003e-05 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239902+0000", 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009000000000001e-05 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:556 4.7 4:e23e5852:::rbd_data.11b59d02cc38.0000000000000074:head [set-alloc-hint object_size 1048576 write_size 1048576,write 736209~50223] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.93851304, 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038485979999999999, 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 556 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264664+0000", 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T12:59:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264757+0000", 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2904000000000004e-05 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264792+0000", 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4437000000000002e-05 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264804+0000", 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.218e-05 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264922+0000", 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118559 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265276+0000", 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035357999999999999 2024-09-16T12:59:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265652+0000", 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037578600000000002 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.267338+0000", 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016864899999999999 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268352+0000", 2024-09-16T12:59:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00101337 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268425+0000", 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3398000000000004e-05 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268509+0000", 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4318000000000001e-05 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:574 4.7 4:e23e5852:::rbd_data.11b59d02cc38.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.890957272, 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051945439999999997, 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 574 2024-09-16T12:59:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3300000000000003e-07 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312221+0000", 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9550000000000002e-06 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312229+0000", 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6159999999999996e-06 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312233+0000", 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8779999999999997e-06 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312886+0000", 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065238099999999995 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313137+0000", 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025079300000000002 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313541+0000", 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040414999999999998 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314738+0000", 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011973839999999999 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317292+0000", 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002554312 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317342+0000", 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9175999999999998e-05 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317411+0000", 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9466000000000002e-05 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:710 4.f 4:f1e0f236:::rbd_data.11b59d02cc38.000000000000008e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 694412~26484] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.688159+0000", 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.515014629, 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034345560000000001, 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 710 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688159+0000", 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688159+0000", 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688162+0000", 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.027e-06 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688195+0000", 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2824999999999998e-05 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688232+0000", 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6745000000000001e-05 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688251+0000", 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9670999999999999e-05 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688371+0000", 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119141 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688594+0000", 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022353100000000001 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688799+0000", 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020529599999999999 2024-09-16T12:59:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.690783+0000", 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019834330000000002 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691370+0000", 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058718900000000001 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691494+0000", 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012406299999999999 2024-09-16T12:59:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691594+0000", 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9635000000000004e-05 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:711 4.4 4:20cc0444:::rbd_data.11b59d02cc38.000000000000008f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 655360~35331] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.514765756999999, 2024-09-16T12:59:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038704080000000001, 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 711 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688410+0000", 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688470+0000", 2024-09-16T12:59:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0034000000000002e-05 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688491+0000", 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605999999999999e-05 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688508+0000", 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7452000000000002e-05 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688639+0000", 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130541 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688984+0000", 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034521699999999998 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.689500+0000", 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000516023 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692011+0000", 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025114349999999998 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692150+0000", 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013872999999999999 2024-09-16T12:59:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692223+0000", 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2917999999999995e-05 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692278+0000", 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5402999999999998e-05 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:715 4.4 4:2570fb00:::rbd_data.11b59d02cc38.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.501875362, 2024-09-16T12:59:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037942140000000002, 2024-09-16T12:59:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 715 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701300+0000", 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9920000000000002e-06 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701311+0000", 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0361e-05 2024-09-16T12:59:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701343+0000", 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2181000000000002e-05 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701361+0000", 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8366999999999999e-05 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701465+0000", 2024-09-16T12:59:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103854 2024-09-16T12:59:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701709+0000", 2024-09-16T12:59:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024390300000000001 2024-09-16T12:59:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702064+0000", 2024-09-16T12:59:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035441500000000001 2024-09-16T12:59:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.704221+0000", 2024-09-16T12:59:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021577609999999998 2024-09-16T12:59:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.704960+0000", 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073849600000000005 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.705039+0000", 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9092999999999996e-05 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.705093+0000", 2024-09-16T12:59:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3791e-05 2024-09-16T12:59:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:719 4.4 4:2105cf60:::rbd_data.11b59d02cc38.0000000000000009:head [set-alloc-hint object_size 1048576 write_size 1048576,write 639060~16300] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.709251+0000", 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.493923101, 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032654839999999999, 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 719 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709251+0000", 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709251+0000", 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709253+0000", 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1969999999999999e-06 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709280+0000", 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7124999999999999e-05 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709315+0000", 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5228000000000002e-05 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709332+0000", 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7028e-05 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709449+0000", 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011715800000000001 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709669+0000", 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021948000000000001 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709887+0000", 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021756799999999999 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712039+0000", 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002152513 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712372+0000", 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033325999999999999 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712459+0000", 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6304000000000006e-05 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712516+0000", 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7623e-05 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:826 4.4 4:22c6e589:::rbd_data.11b59d02cc38.000000000000005a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 196608~7156] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.274023549000001, 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043112890000000003, 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 826 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929153+0000", 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.976e-06 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929175+0000", 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1897999999999999e-05 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929213+0000", 2024-09-16T12:59:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7528999999999999e-05 2024-09-16T12:59:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929230+0000", 2024-09-16T12:59:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7093999999999999e-05 2024-09-16T12:59:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929357+0000", 2024-09-16T12:59:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012754300000000001 2024-09-16T12:59:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929610+0000", 2024-09-16T12:59:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025261899999999999 2024-09-16T12:59:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929979+0000", 2024-09-16T12:59:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036918900000000002 2024-09-16T12:59:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.932855+0000", 2024-09-16T12:59:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028755880000000001 2024-09-16T12:59:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933292+0000", 2024-09-16T12:59:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043741500000000003 2024-09-16T12:59:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933360+0000", 2024-09-16T12:59:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7533999999999994e-05 2024-09-16T12:59:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933462+0000", 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010190400000000001 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:827 4.4 4:2570fb00:::rbd_data.11b59d02cc38.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write 155625~40983] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.273522391, 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045348059999999997, 2024-09-16T12:59:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 827 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929654+0000", 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-06 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929755+0000", 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100955 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929783+0000", 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7902000000000002e-05 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929798+0000", 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5381000000000001e-05 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930965+0000", 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011667310000000001 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.931262+0000", 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029734500000000002 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.931625+0000", 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036316799999999998 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933825+0000", 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021993970000000001 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934061+0000", 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000236095 2024-09-16T12:59:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934138+0000", 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7188999999999996e-05 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934186+0000", 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8303000000000002e-05 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:845 4.7 4:e7d4e431:::rbd_data.11b59d02cc38.0000000000000077:head [set-alloc-hint object_size 1048576 write_size 1048576,write 327680~11175] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.964326+0000", 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.238847614999999, 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003069188, 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 845 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964326+0000", 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964326+0000", 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964327+0000", 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2279999999999999e-06 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964371+0000", 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3460000000000001e-05 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964397+0000", 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999999e-05 2024-09-16T12:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964405+0000", 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3429999999999996e-06 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964562+0000", 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015724700000000001 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964984+0000", 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042120299999999999 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.965431+0000", 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044723799999999998 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967073+0000", 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016422240000000001 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967279+0000", 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020625400000000001 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967346+0000", 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6255999999999997e-05 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967395+0000", 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9774999999999997e-05 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:948 4.f 4:f1e0f236:::rbd_data.11b59d02cc38.000000000000008e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.014613129000001, 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013393758, 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 948 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188563+0000", 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188576+0000", 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3071000000000001e-05 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188611+0000", 2024-09-16T12:59:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4623999999999999e-05 2024-09-16T12:59:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188624+0000", 2024-09-16T12:59:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2958e-05 2024-09-16T12:59:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188732+0000", 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107911 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188873+0000", 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014080700000000001 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.192672+0000", 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037992270000000001 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201725+0000", 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090529549999999997 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201811+0000", 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6328000000000004e-05 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201894+0000", 2024-09-16T12:59:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2991000000000006e-05 2024-09-16T12:59:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201954+0000", 2024-09-16T12:59:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0336e-05 2024-09-16T12:59:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:948 4.f e38/31 ondisk, result = 0)", 2024-09-16T12:59:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.198624+0000", 2024-09-16T12:59:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.004550016, 2024-09-16T12:59:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033460849999999999, 2024-09-16T12:59:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198624+0000", 2024-09-16T12:59:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198624+0000", 2024-09-16T12:59:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198625+0000", 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-06 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198630+0000", 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4650000000000002e-06 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198648+0000", 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7638999999999998e-05 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198656+0000", 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5950000000000001e-06 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201782+0000", 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031262239999999999 2024-09-16T12:59:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201795+0000", 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3121e-05 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201970+0000", 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017486800000000001 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:999 4.f 4:f13e1533:::rbd_data.11b59d02cc38.0000000000000035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.894302866, 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041771059999999999, 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 999 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308872+0000", 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.514e-06 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308880+0000", 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3300000000000001e-06 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308903+0000", 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3017999999999999e-05 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308915+0000", 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2636000000000001e-05 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309000+0000", 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4263000000000003e-05 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309082+0000", 2024-09-16T12:59:33.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2443999999999999e-05 2024-09-16T12:59:33.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:33.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309190+0000", 2024-09-16T12:59:33.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010767700000000001 2024-09-16T12:59:33.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.310638+0000", 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001448509 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.312936+0000", 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002297669 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.312990+0000", 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3494e-05 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.313048+0000", 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8551999999999998e-05 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1164 4.2 4:400c9634:::rbd_header.11b53140b009:head [watch watch cookie 140717154602240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.439318928, 2024-09-16T12:59:33.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0048381040000000002, 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1164 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:33.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763856+0000", 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.096e-06 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763861+0000", 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3199999999999999e-06 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763877+0000", 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.537e-05 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763885+0000", 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.687e-06 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763943+0000", 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7243000000000001e-05 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764001+0000", 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8214000000000002e-05 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764123+0000", 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012186899999999999 2024-09-16T12:59:33.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764972+0000", 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084946000000000002 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768511+0000", 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035386139999999998 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768579+0000", 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8584999999999995e-05 2024-09-16T12:59:33.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768693+0000", 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113646 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1268 4.2 4:40143cfe:::rbd_data.11b53140b009.00000000000000d2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.045972+0000", 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.157201776000001, 2024-09-16T12:59:33.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030905780000000001, 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1268 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045972+0000", 2024-09-16T12:59:33.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045972+0000", 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045975+0000", 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4429999999999999e-06 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045991+0000", 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5475e-05 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046028+0000", 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7205e-05 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046046+0000", 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7520000000000002e-05 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046145+0000", 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9779999999999997e-05 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046385+0000", 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023985099999999999 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046700+0000", 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000314452 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.047681+0000", 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098164099999999989 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049004+0000", 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013222539999999999 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049036+0000", 2024-09-16T12:59:33.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2174000000000001e-05 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049063+0000", 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6783000000000001e-05 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1269 4.7 4:e145092b:::rbd_data.11b53140b009.00000000000000d3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.154145277, 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035041009999999999, 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1269 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:33.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2699999999999999e-07 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049033+0000", 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629999999999997e-06 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049041+0000", 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9920000000000007e-06 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049056+0000", 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4344e-05 2024-09-16T12:59:33.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049131+0000", 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049343+0000", 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021202300000000001 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:33.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049705+0000", 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000362529 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.050871+0000", 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001165391 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052300+0000", 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014292459999999999 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052430+0000", 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013032399999999999 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052533+0000", 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102215 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1287 4.7 4:e145092b:::rbd_data.11b53140b009.00000000000000d3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:33.319 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.13830389, 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042846999999999998, 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1287 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064871+0000", 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-07 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064875+0000", 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8029999999999996e-06 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064889+0000", 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3665e-05 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064906+0000", 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7343e-05 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064971+0000", 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5147999999999994e-05 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.065076+0000", 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105004 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.065203+0000", 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126451 2024-09-16T12:59:33.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.066904+0000", 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017010580000000001 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069097+0000", 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021933299999999998 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069130+0000", 2024-09-16T12:59:33.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3161999999999999e-05 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069155+0000", 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4161000000000001e-05 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1306 4.12 4:4f3fb25c:::rbd_data.11b53140b009.00000000000000c9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 147456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.116173996000001, 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038175800000000001, 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1306 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087001+0000", 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7699999999999992e-07 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087006+0000", 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5010000000000004e-06 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087023+0000", 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.662e-05 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087029+0000", 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4860000000000002e-06 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087105+0000", 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5265000000000005e-05 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087480+0000", 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037499899999999999 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087905+0000", 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042498600000000003 2024-09-16T12:59:33.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.089239+0000", 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013342930000000001 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090707+0000", 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001468235 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090771+0000", 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3553000000000006e-05 2024-09-16T12:59:33.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090817+0000", 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6665000000000001e-05 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1307 4.7 4:e4cf6733:::rbd_data.11b53140b009.00000000000000ca:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 147456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.116117968999999, 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003717405, 2024-09-16T12:59:33.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1307 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087061+0000", 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8919999999999999e-06 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087069+0000", 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1470000000000001e-06 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087074+0000", 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2400000000000001e-06 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087159+0000", 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5765000000000003e-05 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087484+0000", 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032450900000000001 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087913+0000", 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042913400000000002 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.089457+0000", 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001544028 2024-09-16T12:59:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090619+0000", 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011616090000000001 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090710+0000", 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1416999999999996e-05 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090773+0000", 2024-09-16T12:59:33.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3089999999999994e-05 2024-09-16T12:59:33.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.337 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:33.337 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:33.337 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:33.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:33.339 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:33.339 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:33.335+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:33.440 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:33.431+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:33.490 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:33.491 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:33.503 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:33.521 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:33.541 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:33.535+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:33.546 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:33.563 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:33.578 INFO:teuthology.orchestra.run.smithi042.stdout:1173 punch from 0x2549999 to 0x254c157, (0x27be bytes) 2024-09-16T12:59:33.581 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:33.584 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:33.585 INFO:teuthology.orchestra.run.smithi042.stdout:1176 write 0x68ccb2a thru 0x68cf71e (0x2bf5 bytes) 2024-09-16T12:59:33.593 INFO:teuthology.orchestra.run.smithi042.stdout:1177 writesame 0x16b59d1 thru 0x16ba6db data_size 0xb(0x4d0b bytes) 2024-09-16T12:59:33.604 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:33.604 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:33.606 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:33.615 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:33.617 INFO:teuthology.orchestra.run.smithi042.stdout:1178 read 0x537139 thru 0x53f50b (0x83d3 bytes) 2024-09-16T12:59:33.617 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:33.617 INFO:teuthology.orchestra.run.smithi042.stdout:1181 read 0x46eaec7 thru 0x46f72ac (0xc3e6 bytes) 2024-09-16T12:59:33.618 INFO:teuthology.orchestra.run.smithi042.stdout:1182 punch from 0x8bce19 to 0x8c4abe, (0x7ca5 bytes) 2024-09-16T12:59:33.628 INFO:teuthology.orchestra.run.smithi042.stdout:1183 write 0xae3577d thru 0xae388c1 (0x3145 bytes) 2024-09-16T12:59:33.632 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:33.633 INFO:teuthology.orchestra.run.smithi042.stdout:1184 read 0xef8f47 thru 0xf07bbe (0xec78 bytes) 2024-09-16T12:59:33.634 INFO:teuthology.orchestra.run.smithi042.stdout:1186 read 0x4f79d7c thru 0x4f85e1e (0xc0a3 bytes) 2024-09-16T12:59:33.636 INFO:teuthology.orchestra.run.smithi042.stdout:1187 punch from 0x4838b06 to 0x483b590, (0x2a8a bytes) 2024-09-16T12:59:33.636 INFO:teuthology.orchestra.run.smithi042.stdout:1189 punch from 0x72c0453 to 0x72c06e9, (0x296 bytes) 2024-09-16T12:59:33.636 INFO:teuthology.orchestra.run.smithi042.stdout:1190 flatten 2024-09-16T12:59:33.642 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:33.635+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:33.661 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:33.702 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:33.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:33.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:33.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:33.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:33.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:33.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:33.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:33.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:33.711 INFO:teuthology.orchestra.run.smithi042.stdout:1191 write 0xd04e1dd thru 0xd0561ab (0x7fcf bytes) 2024-09-16T12:59:33.718 INFO:teuthology.orchestra.run.smithi042.stdout:1192 punch from 0x8611bde to 0x8612d4c, (0x116e bytes) 2024-09-16T12:59:33.718 INFO:teuthology.orchestra.run.smithi042.stdout:1193 write 0xf6b3e9 thru 0xf7aad7 (0xf6ef bytes) 2024-09-16T12:59:33.723 INFO:teuthology.orchestra.run.smithi042.stdout:1194 write 0x939623b thru 0x939c385 (0x614b bytes) 2024-09-16T12:59:33.730 INFO:teuthology.orchestra.run.smithi042.stdout:1195 read 0x5b4ab0a thru 0x5b4d4f2 (0x29e9 bytes) 2024-09-16T12:59:33.731 INFO:teuthology.orchestra.run.smithi042.stdout:1196 read 0x710d0e3 thru 0x71114e1 (0x43ff bytes) 2024-09-16T12:59:33.731 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:33.731 INFO:teuthology.orchestra.run.smithi042.stdout:1201 write 0x7d07f2d thru 0x7d16375 (0xe449 bytes) 2024-09-16T12:59:33.740 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:33.740 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:33.742 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:33.735+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:33.749 INFO:teuthology.orchestra.run.smithi042.stdout:1202 trunc from 0xd0561ac to 0x7180c70 2024-09-16T12:59:33.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:33.771 INFO:teuthology.orchestra.run.smithi042.stdout:1203 write 0x4fdf775 thru 0x4fe4e15 (0x56a1 bytes) 2024-09-16T12:59:33.794 INFO:teuthology.orchestra.run.smithi042.stdout:1204 write 0xbe60430 thru 0xbe6e1e3 (0xddb4 bytes) 2024-09-16T12:59:33.799 INFO:teuthology.orchestra.run.smithi042.stdout:1206 write 0xa1aa29c thru 0xa1b0b3c (0x68a1 bytes) 2024-09-16T12:59:33.806 INFO:teuthology.orchestra.run.smithi042.stdout:1209 writesame 0x79269fe thru 0x7935bf6 data_size 0xd(0xf1f9 bytes) 2024-09-16T12:59:33.816 INFO:teuthology.orchestra.run.smithi042.stdout:1210 writesame 0x627693f thru 0x6281d39 data_size 0x5f(0xb3fb bytes) 2024-09-16T12:59:33.823 INFO:teuthology.orchestra.run.smithi042.stdout:1212 writesame 0x8799c3e thru 0x87a9ba8 data_size 0x1(0xff6b bytes) 2024-09-16T12:59:33.843 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:33.835+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:33.868 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:33.868 INFO:teuthology.orchestra.run.smithi042.stdout:1214 write 0x6206a93 thru 0x620b783 (0x4cf1 bytes) 2024-09-16T12:59:33.872 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:33.872 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:33.872 INFO:teuthology.orchestra.run.smithi042.stdout:1223 writesame 0xa7cdc79 thru 0xa7ddac1 data_size 0x9(0xfe49 bytes) 2024-09-16T12:59:33.876 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:33.876 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:33.884 INFO:teuthology.orchestra.run.smithi042.stdout:1224 read 0x5a4d694 thru 0x5a5b4d4 (0xde41 bytes) 2024-09-16T12:59:33.884 INFO:teuthology.orchestra.run.smithi042.stdout:1225 punch from 0x31353b4 to 0x3142b5e, (0xd7aa bytes) 2024-09-16T12:59:33.884 INFO:teuthology.orchestra.run.smithi042.stdout:1226 write 0x5840902 thru 0x5849a65 (0x9164 bytes) 2024-09-16T12:59:33.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:33.887 INFO:teuthology.orchestra.run.smithi042.stdout:1227 write 0x5c3c077 thru 0x5c41c49 (0x5bd3 bytes) 2024-09-16T12:59:33.890 INFO:teuthology.orchestra.run.smithi042.stdout:1229 read 0xb4d3500 thru 0xb4d3ac7 (0x5c8 bytes) 2024-09-16T12:59:33.891 INFO:teuthology.orchestra.run.smithi042.stdout:1232 write 0x137bab thru 0x1449a2 (0xcdf8 bytes) 2024-09-16T12:59:33.897 INFO:teuthology.orchestra.run.smithi042.stdout:1233 write 0x6a37aac thru 0x6a3a255 (0x27aa bytes) 2024-09-16T12:59:33.925 INFO:teuthology.orchestra.run.smithi042.stdout:1235 trunc from 0xbe6e1e4 to 0x2f60d 2024-09-16T12:59:33.930 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T12:59:33.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T12:59:33.943 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:33.939+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:33.962 INFO:teuthology.orchestra.run.smithi042.stdout:1236 read 0xdfeb thru 0xf219 (0x122f bytes) 2024-09-16T12:59:33.974 INFO:teuthology.orchestra.run.smithi042.stdout:1237 writesame 0x155a00d thru 0x155e202 data_size 0x1(0x41f6 bytes) 2024-09-16T12:59:33.974 INFO:teuthology.orchestra.run.smithi042.stdout:1238 read 0x108ec2e thru 0x108f560 (0x933 bytes) 2024-09-16T12:59:33.995 INFO:teuthology.orchestra.run.smithi042.stdout:1239 writesame 0x7e81937 thru 0x7e90d9c data_size 0x6d(0xf466 bytes) 2024-09-16T12:59:33.995 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:33.997 INFO:teuthology.orchestra.run.smithi042.stdout:1242 trunc from 0x7e90d9d to 0x9732c70 2024-09-16T12:59:34.000 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:34.000 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:34.006 INFO:teuthology.orchestra.run.smithi042.stdout:1243 write 0x9441aeb thru 0x94501af (0xe6c5 bytes) 2024-09-16T12:59:34.010 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:34.014 INFO:teuthology.orchestra.run.smithi042.stdout:1244 read 0x3abeb02 thru 0x3ac9f2a (0xb429 bytes) 2024-09-16T12:59:34.014 INFO:teuthology.orchestra.run.smithi042.stdout:1246 punch from 0x4134f3b to 0x413b753, (0x6818 bytes) 2024-09-16T12:59:34.014 INFO:teuthology.orchestra.run.smithi042.stdout:1247 writesame 0x234c40b thru 0x2352a64 data_size 0x21(0x665a bytes) 2024-09-16T12:59:34.020 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.021 INFO:teuthology.orchestra.run.smithi042.stdout:1249 writesame 0x2d683ef thru 0x2d72031 data_size 0x1(0x9c43 bytes) 2024-09-16T12:59:34.044 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:34.039+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:34.050 INFO:teuthology.orchestra.run.smithi042.stdout:1251 read 0x30f96ee thru 0x31002ea (0x6bfd bytes) 2024-09-16T12:59:34.050 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.050 INFO:teuthology.orchestra.run.smithi042.stdout:1253 write 0x5bdfbd1 thru 0x5be9d86 (0xa1b6 bytes) 2024-09-16T12:59:34.054 INFO:teuthology.orchestra.run.smithi042.stdout:1255 read 0x12be76d thru 0x12cd75e (0xeff2 bytes) 2024-09-16T12:59:34.054 INFO:teuthology.orchestra.run.smithi042.stdout:1256 write 0x5656db5 thru 0x5662a1e (0xbc6a bytes) 2024-09-16T12:59:34.060 INFO:teuthology.orchestra.run.smithi042.stdout:1257 write 0x6255f4f thru 0x6261d66 (0xbe18 bytes) 2024-09-16T12:59:34.065 INFO:teuthology.orchestra.run.smithi042.stdout:1258 read 0xdc98ac thru 0xdd53f4 (0xbb49 bytes) 2024-09-16T12:59:34.083 INFO:teuthology.orchestra.run.smithi042.stdout:1259 writesame 0xebb2ec7 thru 0xebb6ab9 data_size 0x1(0x3bf3 bytes) 2024-09-16T12:59:34.083 INFO:teuthology.orchestra.run.smithi042.stdout:1260 read 0x8c58b4 thru 0x8c6d5d (0x14aa bytes) 2024-09-16T12:59:34.083 INFO:teuthology.orchestra.run.smithi042.stdout:1263 write 0x1708ced thru 0x1714e40 (0xc154 bytes) 2024-09-16T12:59:34.089 INFO:teuthology.orchestra.run.smithi042.stdout:1264 write 0xcb6b967 thru 0xcb7a259 (0xe8f3 bytes) 2024-09-16T12:59:34.096 INFO:teuthology.orchestra.run.smithi042.stdout:1265 writesame 0x2138b16 thru 0x2145c5f data_size 0x59(0xd14a bytes) 2024-09-16T12:59:34.099 INFO:teuthology.orchestra.run.smithi042.stdout:1266 write 0x7a80eb0 thru 0x7a814ba (0x60b bytes) 2024-09-16T12:59:34.106 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.106 INFO:teuthology.orchestra.run.smithi042.stdout:1268 trunc from 0xebb6aba to 0xe80d909 2024-09-16T12:59:34.124 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:34.124 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:34.128 INFO:teuthology.orchestra.run.smithi042.stdout:1269 trunc from 0xe80d909 to 0x822df30 2024-09-16T12:59:34.136 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:34.170 INFO:teuthology.orchestra.run.smithi042.stdout:1271 write 0xe1e7433 thru 0xe1ee9f7 (0x75c5 bytes) 2024-09-16T12:59:34.171 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:34.163+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:34.177 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.177 INFO:teuthology.orchestra.run.smithi042.stdout:1274 writesame 0x12bde6b thru 0x12c4014 data_size 0x1b(0x61aa bytes) 2024-09-16T12:59:34.184 INFO:teuthology.orchestra.run.smithi042.stdout:1276 punch from 0x5453c47 to 0x5458d81, (0x513a bytes) 2024-09-16T12:59:34.184 INFO:teuthology.orchestra.run.smithi042.stdout:1278 write 0x3263932 thru 0x326b7b6 (0x7e85 bytes) 2024-09-16T12:59:34.187 INFO:teuthology.orchestra.run.smithi042.stdout:1279 writesame 0x34ac4fd thru 0x34bad47 data_size 0x1(0xe84b bytes) 2024-09-16T12:59:34.227 INFO:teuthology.orchestra.run.smithi042.stdout:1280 writesame 0xd2b7f26 thru 0xd2bcff2 data_size 0x23(0x50cd bytes) 2024-09-16T12:59:34.234 INFO:teuthology.orchestra.run.smithi042.stdout:1281 punch from 0x389c94f to 0x38a41a0, (0x7851 bytes) 2024-09-16T12:59:34.234 INFO:teuthology.orchestra.run.smithi042.stdout:1282 write 0x58cd308 thru 0x58cf869 (0x2562 bytes) 2024-09-16T12:59:34.237 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.237 INFO:teuthology.orchestra.run.smithi042.stdout:1285 writesame 0x3e2e38b thru 0x3e3b6ea data_size 0x59(0xd360 bytes) 2024-09-16T12:59:34.240 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:34.240 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:34.243 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.243 INFO:teuthology.orchestra.run.smithi042.stdout:1288 write 0x9626227 thru 0x962bd3c (0x5b16 bytes) 2024-09-16T12:59:34.245 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:34.235+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:34.249 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.249 INFO:teuthology.orchestra.run.smithi042.stdout:1293 read 0x772bfce thru 0x773a675 (0xe6a8 bytes) 2024-09-16T12:59:34.249 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.249 INFO:teuthology.orchestra.run.smithi042.stdout:1296 writesame 0x873b499 thru 0x873e530 data_size 0x14(0x3098 bytes) 2024-09-16T12:59:34.250 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:34.252 INFO:teuthology.orchestra.run.smithi042.stdout:1299 write 0xa1cc752 thru 0xa1cf50c (0x2dbb bytes) 2024-09-16T12:59:34.257 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.257 INFO:teuthology.orchestra.run.smithi042.stdout:1301 punch from 0x98966b7 to 0x98a2f8c, (0xc8d5 bytes) 2024-09-16T12:59:34.257 INFO:teuthology.orchestra.run.smithi042.stdout:1302 write 0xcc544c5 thru 0xcc6029d (0xbdd9 bytes) 2024-09-16T12:59:34.263 INFO:teuthology.orchestra.run.smithi042.stdout:1303 writesame 0x5a08cf7 thru 0x5a0f648 data_size 0xd(0x6952 bytes) 2024-09-16T12:59:34.268 INFO:teuthology.orchestra.run.smithi042.stdout:1305 read 0x8c843b2 thru 0x8c8a1fb (0x5e4a bytes) 2024-09-16T12:59:34.268 INFO:teuthology.orchestra.run.smithi042.stdout:1306 read 0x104e3de thru 0x10559e4 (0x7607 bytes) 2024-09-16T12:59:34.269 INFO:teuthology.orchestra.run.smithi042.stdout:1307 write 0x570c796 thru 0x5712b4f (0x63ba bytes) 2024-09-16T12:59:34.275 INFO:teuthology.orchestra.run.smithi042.stdout:1309 read 0x838aba9 thru 0x8397883 (0xccdb bytes) 2024-09-16T12:59:34.275 INFO:teuthology.orchestra.run.smithi042.stdout:1311 punch from 0xba0746f to 0xba12d4f, (0xb8e0 bytes) 2024-09-16T12:59:34.275 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.276 INFO:teuthology.orchestra.run.smithi042.stdout:1313 trunc from 0xe1ee9f8 to 0xe139da6 2024-09-16T12:59:34.287 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.304 INFO:teuthology.orchestra.run.smithi042.stdout:1316 trunc from 0xe139da6 to 0x4af667e 2024-09-16T12:59:34.328 INFO:teuthology.orchestra.run.smithi042.stdout:1317 punch from 0x2339483 to 0x233d092, (0x3c0f bytes) 2024-09-16T12:59:34.333 INFO:teuthology.orchestra.run.smithi042.stdout:1318 trunc from 0x4af667e to 0x22ac00c 2024-09-16T12:59:34.347 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:34.343+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:34.347 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:34.347 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:1321 punch from 0xc19a6f to 0xc253b8, (0xb949 bytes) 2024-09-16T12:59:34.354 INFO:teuthology.orchestra.run.smithi042.stdout:1322 read 0x5f7a73 thru 0x6042cd (0xc85b bytes) 2024-09-16T12:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.355 INFO:teuthology.orchestra.run.smithi042.stdout:1324 read 0x1f6554d thru 0x1f6cb12 (0x75c6 bytes) 2024-09-16T12:59:34.360 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:34.360 INFO:teuthology.orchestra.run.smithi042.stdout:1325 trunc from 0x22ac00c to 0x45f5d88 2024-09-16T12:59:34.381 INFO:teuthology.orchestra.run.smithi042.stdout:1326 trunc from 0x45f5d88 to 0xfbb0c3 2024-09-16T12:59:34.405 INFO:teuthology.orchestra.run.smithi042.stdout:1327 punch from 0xc729fe to 0xc780a7, (0x56a9 bytes) 2024-09-16T12:59:34.405 INFO:teuthology.orchestra.run.smithi042.stdout:1329 read 0x629266 thru 0x6362ae (0xd049 bytes) 2024-09-16T12:59:34.406 INFO:teuthology.orchestra.run.smithi042.stdout:1330 read 0x13207e thru 0x13d443 (0xb3c6 bytes) 2024-09-16T12:59:34.407 INFO:teuthology.orchestra.run.smithi042.stdout:1331 trunc from 0xfbb0c3 to 0x154eae2 2024-09-16T12:59:34.419 INFO:teuthology.orchestra.run.smithi042.stdout:1334 read 0x23bea6 thru 0x23d9dd (0x1b38 bytes) 2024-09-16T12:59:34.419 INFO:teuthology.orchestra.run.smithi042.stdout:1335 punch from 0x3d935d to 0x3e5bf8, (0xc89b bytes) 2024-09-16T12:59:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:1336 trunc from 0x154eae2 to 0x84fc52c 2024-09-16T12:59:34.447 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:34.443+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:34.448 INFO:teuthology.orchestra.run.smithi042.stdout:1339 write 0x824a318 thru 0x8254ca5 (0xa98e bytes) 2024-09-16T12:59:34.457 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:34.457 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:34.467 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:34.472 INFO:teuthology.orchestra.run.smithi042.stdout:1341 write 0xb9e3f0e thru 0xb9f0fd2 (0xd0c5 bytes) 2024-09-16T12:59:34.478 INFO:teuthology.orchestra.run.smithi042.stdout:1342 writesame 0xb247b7e thru 0xb255baf data_size 0x1(0xe032 bytes) 2024-09-16T12:59:34.515 INFO:teuthology.orchestra.run.smithi042.stdout:1343 read 0x4d0e6e0 thru 0x4d1a0a5 (0xb9c6 bytes) 2024-09-16T12:59:34.515 INFO:teuthology.orchestra.run.smithi042.stdout:1344 read 0x10e6017 thru 0x10ec795 (0x677f bytes) 2024-09-16T12:59:34.515 INFO:teuthology.orchestra.run.smithi042.stdout:1345 read 0x4ddc27f thru 0x4de0d31 (0x4ab3 bytes) 2024-09-16T12:59:34.515 INFO:teuthology.orchestra.run.smithi042.stdout:1346 read 0x84aacf3 thru 0x84b5cac (0xafba bytes) 2024-09-16T12:59:34.515 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.531 INFO:teuthology.orchestra.run.smithi042.stdout:1348 writesame 0xce8c55e thru 0xce93ff3 data_size 0x47(0x7a96 bytes) 2024-09-16T12:59:34.531 INFO:teuthology.orchestra.run.smithi042.stdout:1349 punch from 0x793f2e9 to 0x7941321, (0x2038 bytes) 2024-09-16T12:59:34.531 INFO:teuthology.orchestra.run.smithi042.stdout:1350 write 0x105ac13 thru 0x1061083 (0x6471 bytes) 2024-09-16T12:59:34.545 INFO:teuthology.orchestra.run.smithi042.stdout:1351 write 0xd0e04ca thru 0xd0ef7b7 (0xf2ee bytes) 2024-09-16T12:59:34.548 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:34.543+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:34.551 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.551 INFO:teuthology.orchestra.run.smithi042.stdout:1356 read 0x61eb1dc thru 0x61f2dc7 (0x7bec bytes) 2024-09-16T12:59:34.551 INFO:teuthology.orchestra.run.smithi042.stdout:1357 write 0x84b92 thru 0x90807 (0xbc76 bytes) 2024-09-16T12:59:34.558 INFO:teuthology.orchestra.run.smithi042.stdout:1358 write 0xa198767 thru 0xa1a274a (0x9fe4 bytes) 2024-09-16T12:59:34.563 INFO:teuthology.orchestra.run.smithi042.stdout:1361 write 0x6fc6a09 thru 0x6fc9682 (0x2c7a bytes) 2024-09-16T12:59:34.573 INFO:teuthology.orchestra.run.smithi042.stdout:1362 trunc from 0xd0ef7b8 to 0xe6a1f1f 2024-09-16T12:59:34.611 INFO:teuthology.orchestra.run.smithi042.stdout:1363 trunc from 0xe6a1f1f to 0x270d928 2024-09-16T12:59:34.612 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:34.612 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:34.623 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:34.633 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.633 INFO:teuthology.orchestra.run.smithi042.stdout:1366 read 0xd4dc28 thru 0xd56280 (0x8659 bytes) 2024-09-16T12:59:34.649 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:34.639+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:34.660 INFO:teuthology.orchestra.run.smithi042.stdout:1367 write 0x9e37768 thru 0x9e429d3 (0xb26c bytes) 2024-09-16T12:59:34.667 INFO:teuthology.orchestra.run.smithi042.stdout:1368 read 0x9d223c7 thru 0x9d2af53 (0x8b8d bytes) 2024-09-16T12:59:34.667 INFO:teuthology.orchestra.run.smithi042.stdout:1370 trunc from 0x9e429d4 to 0x9635d0a 2024-09-16T12:59:34.681 INFO:teuthology.orchestra.run.smithi042.stdout:1371 read 0x300d3a3 thru 0x3017918 (0xa576 bytes) 2024-09-16T12:59:34.681 INFO:teuthology.orchestra.run.smithi042.stdout:1372 write 0x3550343 thru 0x35587ee (0x84ac bytes) 2024-09-16T12:59:34.687 INFO:teuthology.orchestra.run.smithi042.stdout:1373 trunc from 0x9635d0a to 0x9609202 2024-09-16T12:59:34.715 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:34.716 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:34.717 INFO:teuthology.orchestra.run.smithi042.stdout:1374 write 0xb46bdbf thru 0xb47563d (0x987f bytes) 2024-09-16T12:59:34.724 INFO:teuthology.orchestra.run.smithi042.stdout:1375 write 0xa8043d5 thru 0xa813282 (0xeeae bytes) 2024-09-16T12:59:34.725 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:34.731 INFO:teuthology.orchestra.run.smithi042.stdout:1377 punch from 0x6d122e0 to 0x6d16628, (0x4348 bytes) 2024-09-16T12:59:34.747 INFO:teuthology.orchestra.run.smithi042.stdout:1378 write 0xd1e4a4d thru 0xd1f021a (0xb7ce bytes) 2024-09-16T12:59:34.750 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:34.743+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:34.754 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.754 INFO:teuthology.orchestra.run.smithi042.stdout:1381 write 0xc729093 thru 0xc734ec9 (0xbe37 bytes) 2024-09-16T12:59:34.760 INFO:teuthology.orchestra.run.smithi042.stdout:1382 write 0x1461e4d thru 0x1466202 (0x43b6 bytes) 2024-09-16T12:59:34.766 INFO:teuthology.orchestra.run.smithi042.stdout:1383 read 0x11d6ef5 thru 0x11e4e68 (0xdf74 bytes) 2024-09-16T12:59:34.768 INFO:teuthology.orchestra.run.smithi042.stdout:1386 read 0x7627e2f thru 0x76303c8 (0x859a bytes) 2024-09-16T12:59:34.768 INFO:teuthology.orchestra.run.smithi042.stdout:1388 write 0x964065d thru 0x964cd88 (0xc72c bytes) 2024-09-16T12:59:34.774 INFO:teuthology.orchestra.run.smithi042.stdout:1389 write 0x23fcd1d thru 0x240282c (0x5b10 bytes) 2024-09-16T12:59:34.804 INFO:teuthology.orchestra.run.smithi042.stdout:1390 trunc from 0xd1f021b to 0x1c4c4ea 2024-09-16T12:59:34.827 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:34.827 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:34.827 INFO:teuthology.orchestra.run.smithi042.stdout:1391 trunc from 0x1c4c4ea to 0x6cdab4 2024-09-16T12:59:34.837 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:34.846 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.846 INFO:teuthology.orchestra.run.smithi042.stdout:1394 punch from 0x42b721 to 0x430acb, (0x53aa bytes) 2024-09-16T12:59:34.851 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:34.847+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:34.873 INFO:teuthology.orchestra.run.smithi042.stdout:1395 write 0x9120195 thru 0x9122a91 (0x28fd bytes) 2024-09-16T12:59:34.880 INFO:teuthology.orchestra.run.smithi042.stdout:1398 read 0x40508ba thru 0x4056921 (0x6068 bytes) 2024-09-16T12:59:34.880 INFO:teuthology.orchestra.run.smithi042.stdout:1400 punch from 0x7fd7254 to 0x7fddbf6, (0x69a2 bytes) 2024-09-16T12:59:34.880 INFO:teuthology.orchestra.run.smithi042.stdout:1404 write 0x8bfa7e5 thru 0x8c06e94 (0xc6b0 bytes) 2024-09-16T12:59:34.886 INFO:teuthology.orchestra.run.smithi042.stdout:1405 writesame 0x6b47d24 thru 0x6b5693c data_size 0x1(0xec19 bytes) 2024-09-16T12:59:34.935 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:34.935 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:34.945 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:34.949 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T12:59:34.954 INFO:teuthology.orchestra.run.smithi042.stdout:1406 write 0xea6581f thru 0xea6c58a (0x6d6c bytes) 2024-09-16T12:59:34.959 INFO:teuthology.orchestra.run.smithi042.stdout:1408 punch from 0xae74953 to 0xae7e3c0, (0x9a6d bytes) 2024-09-16T12:59:34.959 INFO:teuthology.orchestra.run.smithi042.stdout:1409 writesame 0x67cc4aa thru 0x67d1e0f data_size 0x1(0x5966 bytes) 2024-09-16T12:59:34.977 INFO:teuthology.orchestra.run.smithi042.stdout:1410 read 0x6086d37 thru 0x60956f2 (0xe9bc bytes) 2024-09-16T12:59:34.979 INFO:teuthology.orchestra.run.smithi042.stdout:1412 trunc from 0xea6c58b to 0xdd05860 2024-09-16T12:59:34.990 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:34.987+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:34.996 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.996 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:34.996 INFO:teuthology.orchestra.run.smithi042.stdout:1416 read 0xb65f637 thru 0xb6641ae (0x4b78 bytes) 2024-09-16T12:59:34.996 INFO:teuthology.orchestra.run.smithi042.stdout:1417 write 0xa60817 thru 0xa6a233 (0x9a1d bytes) 2024-09-16T12:59:35.003 INFO:teuthology.orchestra.run.smithi042.stdout:1418 punch from 0xd10035f to 0xd10e3cf, (0xe070 bytes) 2024-09-16T12:59:35.004 INFO:teuthology.orchestra.run.smithi042.stdout:1420 write 0x5fa4f4a thru 0x5fb469b (0xf752 bytes) 2024-09-16T12:59:35.010 INFO:teuthology.orchestra.run.smithi042.stdout:1421 punch from 0xca90d29 to 0xca9da70, (0xcd47 bytes) 2024-09-16T12:59:35.010 INFO:teuthology.orchestra.run.smithi042.stdout:1422 read 0xa78b2ca thru 0xa78f838 (0x456f bytes) 2024-09-16T12:59:35.010 INFO:teuthology.orchestra.run.smithi042.stdout:1423 write 0x92c0a79 thru 0x92c897e (0x7f06 bytes) 2024-09-16T12:59:35.016 INFO:teuthology.orchestra.run.smithi042.stdout:1424 write 0x7d68c17 thru 0x7d6c425 (0x380f bytes) 2024-09-16T12:59:35.024 INFO:teuthology.orchestra.run.smithi042.stdout:1425 writesame 0x9593275 thru 0x959bee4 data_size 0x70(0x8c70 bytes) 2024-09-16T12:59:35.031 INFO:teuthology.orchestra.run.smithi042.stdout:1427 punch from 0x24ab882 to 0x24bb117, (0xf895 bytes) 2024-09-16T12:59:35.032 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:35.032 INFO:teuthology.orchestra.run.smithi042.stdout:1429 write 0x66c0fdb thru 0x66d0a94 (0xfaba bytes) 2024-09-16T12:59:35.038 INFO:teuthology.orchestra.run.smithi042.stdout:1430 write 0x6f6e4ab thru 0x6f70acf (0x2625 bytes) 2024-09-16T12:59:35.041 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:35.041 INFO:teuthology.orchestra.run.smithi042.stdout:1434 read 0x2052597 thru 0x2060a8a (0xe4f4 bytes) 2024-09-16T12:59:35.041 INFO:teuthology.orchestra.run.smithi042.stdout:1435 writesame 0xd19fc9e thru 0xd1a2e0b data_size 0x39(0x316e bytes) 2024-09-16T12:59:35.042 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:35.042 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:35.047 INFO:teuthology.orchestra.run.smithi042.stdout:1438 read 0x797163a thru 0x797bd4a (0xa711 bytes) 2024-09-16T12:59:35.047 INFO:teuthology.orchestra.run.smithi042.stdout:1439 writesame 0xcc14269 thru 0xcc20929 data_size 0x29(0xc6c1 bytes) 2024-09-16T12:59:35.052 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:35.053 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:35.047+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:35.054 INFO:teuthology.orchestra.run.smithi042.stdout:1440 write 0xc6e3cb8 thru 0xc6f0e04 (0xd14d bytes) 2024-09-16T12:59:35.061 INFO:teuthology.orchestra.run.smithi042.stdout:1442 read 0xabfab1e thru 0xabfece1 (0x41c4 bytes) 2024-09-16T12:59:35.078 INFO:teuthology.orchestra.run.smithi042.stdout:1444 trunc from 0xdd05860 to 0x7702840 2024-09-16T12:59:35.104 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:35.104 INFO:teuthology.orchestra.run.smithi042.stdout:1447 read 0x2d937b3 thru 0x2d9bf15 (0x8763 bytes) 2024-09-16T12:59:35.104 INFO:teuthology.orchestra.run.smithi042.stdout:1449 read 0x5c7fa83 thru 0x5c84280 (0x47fe bytes) 2024-09-16T12:59:35.105 INFO:teuthology.orchestra.run.smithi042.stdout:1453 read 0x44c49e0 thru 0x44c5410 (0xa31 bytes) 2024-09-16T12:59:35.105 INFO:teuthology.orchestra.run.smithi042.stdout:1456 write 0x5bb6568 thru 0x5bbb0b3 (0x4b4c bytes) 2024-09-16T12:59:35.110 INFO:teuthology.orchestra.run.smithi042.stdout:1457 punch from 0x6b2f854 to 0x6b3741b, (0x7bc7 bytes) 2024-09-16T12:59:35.110 INFO:teuthology.orchestra.run.smithi042.stdout:1459 read 0x38b4e39 thru 0x38c3e45 (0xf00d bytes) 2024-09-16T12:59:35.110 INFO:teuthology.orchestra.run.smithi042.stdout:1460 read 0x20aed52 thru 0x20b02bf (0x156e bytes) 2024-09-16T12:59:35.111 INFO:teuthology.orchestra.run.smithi042.stdout:1462 read 0x547f8f3 thru 0x548b5c5 (0xbcd3 bytes) 2024-09-16T12:59:35.119 INFO:teuthology.orchestra.run.smithi042.stdout:1463 trunc from 0x7702840 to 0xb7e1466 2024-09-16T12:59:35.129 INFO:teuthology.orchestra.run.smithi042.stdout:1465 read 0x2289ac2 thru 0x2291e53 (0x8392 bytes) 2024-09-16T12:59:35.130 INFO:teuthology.orchestra.run.smithi042.stdout:1466 read 0x5bb148 thru 0x5c50ad (0x9f66 bytes) 2024-09-16T12:59:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:1468 trunc from 0xb7e1466 to 0xe4b44da 2024-09-16T12:59:35.150 INFO:teuthology.orchestra.run.smithi042.stdout:1470 read 0xbf95899 thru 0xbfa1f00 (0xc668 bytes) 2024-09-16T12:59:35.150 INFO:teuthology.orchestra.run.smithi042.stdout:1474 read 0x21c8c17 thru 0x21d5d5d (0xd147 bytes) 2024-09-16T12:59:35.150 INFO:teuthology.orchestra.run.smithi042.stdout:1475 read 0x780c90c thru 0x7811386 (0x4a7b bytes) 2024-09-16T12:59:35.151 INFO:teuthology.orchestra.run.smithi042.stdout:1476 write 0xd91384d thru 0xd920fd0 (0xd784 bytes) 2024-09-16T12:59:35.153 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:35.143+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:35.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:35.158 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:1477 write 0xb55e1ef thru 0xb56d864 (0xf676 bytes) 2024-09-16T12:59:35.165 INFO:teuthology.orchestra.run.smithi042.stdout:1478 punch from 0x936a336 to 0x936b2e1, (0xfab bytes) 2024-09-16T12:59:35.166 INFO:teuthology.orchestra.run.smithi042.stdout:1480 read 0x998920a thru 0x9998d52 (0xfb49 bytes) 2024-09-16T12:59:35.166 INFO:teuthology.orchestra.run.smithi042.stdout:1481 read 0xb4131c9 thru 0xb41e3c1 (0xb1f9 bytes) 2024-09-16T12:59:35.166 INFO:teuthology.orchestra.run.smithi042.stdout:1482 punch from 0xbc5ffb0 to 0xbc6cc31, (0xcc81 bytes) 2024-09-16T12:59:35.166 INFO:teuthology.orchestra.run.smithi042.stdout:1483 read 0x474a48f thru 0x4756aa5 (0xc617 bytes) 2024-09-16T12:59:35.166 INFO:teuthology.orchestra.run.smithi042.stdout:1486 write 0x76797b9 thru 0x7685a46 (0xc28e bytes) 2024-09-16T12:59:35.167 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:35.173 INFO:teuthology.orchestra.run.smithi042.stdout:1487 write 0x5976423 thru 0x5984b21 (0xe6ff bytes) 2024-09-16T12:59:35.180 INFO:teuthology.orchestra.run.smithi042.stdout:1490 write 0x2f7ec7d thru 0x2f8475e (0x5ae2 bytes) 2024-09-16T12:59:35.187 INFO:teuthology.orchestra.run.smithi042.stdout:1491 write 0x6b30af8 thru 0x6b38be3 (0x80ec bytes) 2024-09-16T12:59:35.217 INFO:teuthology.orchestra.run.smithi042.stdout:1492 trunc from 0xe4b44da to 0x1195154 2024-09-16T12:59:35.252 INFO:teuthology.orchestra.run.smithi042.stdout:1493 trunc from 0x1195154 to 0x6194225 2024-09-16T12:59:35.254 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:35.243+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:35.261 INFO:teuthology.orchestra.run.smithi042.stdout:1496 punch from 0x232b3b to 0x237a46, (0x4f0b bytes) 2024-09-16T12:59:35.261 INFO:teuthology.orchestra.run.smithi042.stdout:1499 clone 10 order 25 su 524288 sc 10 2024-09-16T12:59:35.262 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:35.262 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:35.271 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:35.355 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:35.347+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:35.364 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:35.364 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:35.373 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:35.456 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:35.451+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:35.473 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:35.474 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:35.483 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:35.558 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:35.555+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:35.593 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:35.593 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:35.603 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:35.658 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:35.651+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:35.707 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:35.707 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:35.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:35.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:35.759 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:35.755+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:35.819 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:35.819 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:35.829 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:35.860 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:35.851+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:35.919 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:35.919 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:35.919 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:35.919 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:35.928 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:35.932 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:35.959 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:35.960 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:35.955+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:35.982 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:36.007 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:36.009 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:36.009 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:36.009 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:36.009 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:36.009 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:36.009 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:36.009 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:36.009 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:36.034 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:36.034 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:36.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:36.090 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:36.083+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:36.162 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:36.155+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:36.177 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-16T12:59:36.197 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:346 4.2 4:4550e1e5:::rbd_id.image_client.0-clone1:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.34103494, 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003768212, 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.198 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 346 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850713+0000", 2024-09-16T12:59:36.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6270000000000001e-06 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850722+0000", 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2169999999999994e-06 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850739+0000", 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7005000000000001e-05 2024-09-16T12:59:36.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850758+0000", 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9502999999999999e-05 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850826+0000", 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8032000000000002e-05 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850959+0000", 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013249600000000001 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.851164+0000", 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020556 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.852079+0000", 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091475699999999998 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854331+0000", 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022523040000000001 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854400+0000", 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8356999999999996e-05 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854480+0000", 2024-09-16T12:59:36.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0353999999999997e-05 2024-09-16T12:59:36.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.205 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.205 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:543 4.7 4:e655f1b3:::rbd_data.11b59d02cc38.000000000000004c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 367827~25389] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.205 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:36.205 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.956428527, 2024-09-16T12:59:36.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004583953, 2024-09-16T12:59:36.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.206 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 543 2024-09-16T12:59:36.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.206 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235321+0000", 2024-09-16T12:59:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.117e-06 2024-09-16T12:59:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235354+0000", 2024-09-16T12:59:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2962000000000001e-05 2024-09-16T12:59:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235394+0000", 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9954999999999999e-05 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235409+0000", 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4756999999999999e-05 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235553+0000", 2024-09-16T12:59:36.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000144284 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235809+0000", 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025601299999999998 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.236081+0000", 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000272093 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237888+0000", 2024-09-16T12:59:36.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806298 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239756+0000", 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001867884 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239842+0000", 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6581000000000003e-05 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239902+0000", 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009000000000001e-05 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:556 4.7 4:e23e5852:::rbd_data.11b59d02cc38.0000000000000074:head [set-alloc-hint object_size 1048576 write_size 1048576,write 736209~50223] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.927085975000001, 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038485979999999999, 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 556 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264664+0000", 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264757+0000", 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2904000000000004e-05 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264792+0000", 2024-09-16T12:59:36.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4437000000000002e-05 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264804+0000", 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.218e-05 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264922+0000", 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118559 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265276+0000", 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035357999999999999 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265652+0000", 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037578600000000002 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.267338+0000", 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016864899999999999 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268352+0000", 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00101337 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268425+0000", 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3398000000000004e-05 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268509+0000", 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4318000000000001e-05 2024-09-16T12:59:36.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:574 4.7 4:e23e5852:::rbd_data.11b59d02cc38.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.879530206999998, 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051945439999999997, 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 574 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3300000000000003e-07 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312221+0000", 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9550000000000002e-06 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312229+0000", 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6159999999999996e-06 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312233+0000", 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8779999999999997e-06 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312886+0000", 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065238099999999995 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313137+0000", 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025079300000000002 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313541+0000", 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040414999999999998 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314738+0000", 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011973839999999999 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317292+0000", 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002554312 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317342+0000", 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9175999999999998e-05 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317411+0000", 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9466000000000002e-05 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:710 4.f 4:f1e0f236:::rbd_data.11b59d02cc38.000000000000008e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 694412~26484] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.688159+0000", 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.503587564, 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034345560000000001, 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 710 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688159+0000", 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688159+0000", 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688162+0000", 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.027e-06 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688195+0000", 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2824999999999998e-05 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688232+0000", 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6745000000000001e-05 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688251+0000", 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9670999999999999e-05 2024-09-16T12:59:36.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688371+0000", 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119141 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688594+0000", 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022353100000000001 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688799+0000", 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020529599999999999 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.690783+0000", 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019834330000000002 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691370+0000", 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058718900000000001 2024-09-16T12:59:36.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691494+0000", 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012406299999999999 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.691594+0000", 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9635000000000004e-05 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:711 4.4 4:20cc0444:::rbd_data.11b59d02cc38.000000000000008f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 655360~35331] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.503338692, 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038704080000000001, 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 711 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688410+0000", 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688470+0000", 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0034000000000002e-05 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688491+0000", 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605999999999999e-05 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688508+0000", 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7452000000000002e-05 2024-09-16T12:59:36.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688639+0000", 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130541 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688984+0000", 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034521699999999998 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:36.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.689500+0000", 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000516023 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692011+0000", 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025114349999999998 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692150+0000", 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013872999999999999 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692223+0000", 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2917999999999995e-05 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692278+0000", 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5402999999999998e-05 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.235 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:715 4.4 4:2570fb00:::rbd_data.11b59d02cc38.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.490448297, 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037942140000000002, 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 715 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701300+0000", 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9920000000000002e-06 2024-09-16T12:59:36.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701311+0000", 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0361e-05 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701343+0000", 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2181000000000002e-05 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701361+0000", 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8366999999999999e-05 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701465+0000", 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103854 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701709+0000", 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024390300000000001 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702064+0000", 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035441500000000001 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.704221+0000", 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021577609999999998 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.704960+0000", 2024-09-16T12:59:36.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073849600000000005 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.705039+0000", 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9092999999999996e-05 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.705093+0000", 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3791e-05 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.241 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:719 4.4 4:2105cf60:::rbd_data.11b59d02cc38.0000000000000009:head [set-alloc-hint object_size 1048576 write_size 1048576,write 639060~16300] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.709251+0000", 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.482496036000001, 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032654839999999999, 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 719 2024-09-16T12:59:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709251+0000", 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709251+0000", 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709253+0000", 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1969999999999999e-06 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709280+0000", 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7124999999999999e-05 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709315+0000", 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5228000000000002e-05 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709332+0000", 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7028e-05 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709449+0000", 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011715800000000001 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709669+0000", 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021948000000000001 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.709887+0000", 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021756799999999999 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712039+0000", 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002152513 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712372+0000", 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033325999999999999 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712459+0000", 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6304000000000006e-05 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.712516+0000", 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7623e-05 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:826 4.4 4:22c6e589:::rbd_data.11b59d02cc38.000000000000005a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 196608~7156] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:36.247 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.262596483999999, 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043112890000000003, 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 826 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929153+0000", 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.976e-06 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929175+0000", 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1897999999999999e-05 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929213+0000", 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7528999999999999e-05 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929230+0000", 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7093999999999999e-05 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929357+0000", 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012754300000000001 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929610+0000", 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025261899999999999 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929979+0000", 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036918900000000002 2024-09-16T12:59:36.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.932855+0000", 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028755880000000001 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933292+0000", 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043741500000000003 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933360+0000", 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7533999999999994e-05 2024-09-16T12:59:36.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933462+0000", 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010190400000000001 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:827 4.4 4:2570fb00:::rbd_data.11b59d02cc38.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write 155625~40983] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.262095326000001, 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045348059999999997, 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 827 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929654+0000", 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-06 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929755+0000", 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100955 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929783+0000", 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7902000000000002e-05 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929798+0000", 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5381000000000001e-05 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930965+0000", 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011667310000000001 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.931262+0000", 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029734500000000002 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.931625+0000", 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036316799999999998 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933825+0000", 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021993970000000001 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934061+0000", 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000236095 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934138+0000", 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7188999999999996e-05 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934186+0000", 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8303000000000002e-05 2024-09-16T12:59:36.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:845 4.7 4:e7d4e431:::rbd_data.11b59d02cc38.0000000000000077:head [set-alloc-hint object_size 1048576 write_size 1048576,write 327680~11175] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.964326+0000", 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.227420550000001, 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003069188, 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 845 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964326+0000", 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964326+0000", 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964327+0000", 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2279999999999999e-06 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964371+0000", 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3460000000000001e-05 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964397+0000", 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999999e-05 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964405+0000", 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3429999999999996e-06 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964562+0000", 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015724700000000001 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.964984+0000", 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042120299999999999 2024-09-16T12:59:36.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.965431+0000", 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044723799999999998 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967073+0000", 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016422240000000001 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967279+0000", 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020625400000000001 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967346+0000", 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6255999999999997e-05 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.967395+0000", 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9774999999999997e-05 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:948 4.f 4:f1e0f236:::rbd_data.11b59d02cc38.000000000000008e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.003186064000001, 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013393758, 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 948 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188563+0000", 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188576+0000", 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3071000000000001e-05 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188611+0000", 2024-09-16T12:59:36.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4623999999999999e-05 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188624+0000", 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2958e-05 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188732+0000", 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107911 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188873+0000", 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014080700000000001 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.192672+0000", 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037992270000000001 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201725+0000", 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090529549999999997 2024-09-16T12:59:36.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201811+0000", 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6328000000000004e-05 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201894+0000", 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2991000000000006e-05 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201954+0000", 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0336e-05 2024-09-16T12:59:36.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:948 4.f e38/31 ondisk, result = 0)", 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.198624+0000", 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.993122951, 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033460849999999999, 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198624+0000", 2024-09-16T12:59:36.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198624+0000", 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198625+0000", 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-06 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198630+0000", 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4650000000000002e-06 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198648+0000", 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7638999999999998e-05 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.198656+0000", 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5950000000000001e-06 2024-09-16T12:59:36.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201782+0000", 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031262239999999999 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201795+0000", 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3121e-05 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201970+0000", 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017486800000000001 2024-09-16T12:59:36.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:999 4.f 4:f13e1533:::rbd_data.11b59d02cc38.0000000000000035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.882875801000001, 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041771059999999999, 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 999 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308872+0000", 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.514e-06 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308880+0000", 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3300000000000001e-06 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308903+0000", 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3017999999999999e-05 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308915+0000", 2024-09-16T12:59:36.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2636000000000001e-05 2024-09-16T12:59:36.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309000+0000", 2024-09-16T12:59:36.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4263000000000003e-05 2024-09-16T12:59:36.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309082+0000", 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2443999999999999e-05 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309190+0000", 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010767700000000001 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.310638+0000", 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001448509 2024-09-16T12:59:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.312936+0000", 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002297669 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.312990+0000", 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3494e-05 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.313048+0000", 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8551999999999998e-05 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1164 4.2 4:400c9634:::rbd_header.11b53140b009:head [watch watch cookie 140717154602240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.427891862999999, 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0048381040000000002, 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1164 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763856+0000", 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.096e-06 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763861+0000", 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3199999999999999e-06 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763877+0000", 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.537e-05 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763885+0000", 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.687e-06 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763943+0000", 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7243000000000001e-05 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764001+0000", 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8214000000000002e-05 2024-09-16T12:59:36.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764123+0000", 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012186899999999999 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764972+0000", 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084946000000000002 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768511+0000", 2024-09-16T12:59:36.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035386139999999998 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768579+0000", 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8584999999999995e-05 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768693+0000", 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113646 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1268 4.2 4:40143cfe:::rbd_data.11b53140b009.00000000000000d2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.045972+0000", 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.145774711000001, 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030905780000000001, 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1268 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.287 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045972+0000", 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045972+0000", 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045975+0000", 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4429999999999999e-06 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.045991+0000", 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5475e-05 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046028+0000", 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7205e-05 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046046+0000", 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7520000000000002e-05 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046145+0000", 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9779999999999997e-05 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046385+0000", 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023985099999999999 2024-09-16T12:59:36.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.046700+0000", 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000314452 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.047681+0000", 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098164099999999989 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049004+0000", 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013222539999999999 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049036+0000", 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2174000000000001e-05 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049063+0000", 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6783000000000001e-05 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.292 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1269 4.7 4:e145092b:::rbd_data.11b53140b009.00000000000000d3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.142718211999998, 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035041009999999999, 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.293 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1269 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2699999999999999e-07 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049033+0000", 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629999999999997e-06 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049041+0000", 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9920000000000007e-06 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049056+0000", 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4344e-05 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049131+0000", 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049343+0000", 2024-09-16T12:59:36.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021202300000000001 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049705+0000", 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000362529 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.050871+0000", 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001165391 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052300+0000", 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014292459999999999 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052430+0000", 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013032399999999999 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052533+0000", 2024-09-16T12:59:36.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102215 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1287 4.7 4:e145092b:::rbd_data.11b53140b009.00000000000000d3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.126876825, 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042846999999999998, 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.299 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1287 2024-09-16T12:59:36.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.300 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:36.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064871+0000", 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-07 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064875+0000", 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8029999999999996e-06 2024-09-16T12:59:36.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064889+0000", 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3665e-05 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064906+0000", 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7343e-05 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064971+0000", 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5147999999999994e-05 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.065076+0000", 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105004 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.065203+0000", 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126451 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.066904+0000", 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017010580000000001 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069097+0000", 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021933299999999998 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069130+0000", 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3161999999999999e-05 2024-09-16T12:59:36.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069155+0000", 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4161000000000001e-05 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1306 4.12 4:4f3fb25c:::rbd_data.11b53140b009.00000000000000c9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 147456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.104746931000001, 2024-09-16T12:59:36.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038175800000000001, 2024-09-16T12:59:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1306 2024-09-16T12:59:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087001+0000", 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7699999999999992e-07 2024-09-16T12:59:36.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087006+0000", 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5010000000000004e-06 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087023+0000", 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.662e-05 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087029+0000", 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4860000000000002e-06 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087105+0000", 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5265000000000005e-05 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087480+0000", 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037499899999999999 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087905+0000", 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042498600000000003 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.089239+0000", 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013342930000000001 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090707+0000", 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001468235 2024-09-16T12:59:36.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090771+0000", 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3553000000000006e-05 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090817+0000", 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6665000000000001e-05 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1307 4.7 4:e4cf6733:::rbd_data.11b53140b009.00000000000000ca:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 147456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.104690904000002, 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003717405, 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1307 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.313 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-09-16T12:59:36.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087061+0000", 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8919999999999999e-06 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087069+0000", 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1470000000000001e-06 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087074+0000", 2024-09-16T12:59:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2400000000000001e-06 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087159+0000", 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5765000000000003e-05 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087484+0000", 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032450900000000001 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087913+0000", 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042913400000000002 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.089457+0000", 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001544028 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090619+0000", 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011616090000000001 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090710+0000", 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1416999999999996e-05 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090773+0000", 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3089999999999994e-05 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.318 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:36.320 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:36.321 INFO:teuthology.orchestra.run.smithi042.stdout:1500 read 0x48bb0c3 thru 0x48c9f1e (0xee5c bytes) 2024-09-16T12:59:36.321 INFO:teuthology.orchestra.run.smithi042.stdout:1501 trunc from 0x6194225 to 0x164add3 2024-09-16T12:59:36.321 INFO:teuthology.orchestra.run.smithi042.stdout:1502 read 0x336cbc thru 0x33fb26 (0x8e6b bytes) 2024-09-16T12:59:36.321 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:36.311+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:36.350 INFO:teuthology.orchestra.run.smithi042.stdout:1503 write 0xe265576 thru 0xe26a48b (0x4f16 bytes) 2024-09-16T12:59:36.353 INFO:teuthology.orchestra.run.smithi042.stdout:1504 write 0x95d1f98 thru 0x95dca6c (0xaad5 bytes) 2024-09-16T12:59:36.356 INFO:teuthology.orchestra.run.smithi042.stdout:1505 write 0xad1d6d0 thru 0xad2d56e (0xfe9f bytes) 2024-09-16T12:59:36.360 INFO:teuthology.orchestra.run.smithi042.stdout:1506 read 0x89df584 thru 0x89e27e9 (0x3266 bytes) 2024-09-16T12:59:36.373 INFO:teuthology.orchestra.run.smithi042.stdout:1507 write 0xe81d001 thru 0xe829d8a (0xcd8a bytes) 2024-09-16T12:59:36.386 INFO:teuthology.orchestra.run.smithi042.stdout:1509 trunc from 0xe829d8b to 0x957f371 2024-09-16T12:59:36.403 INFO:teuthology.orchestra.run.smithi042.stdout:1510 writesame 0x62dd8c5 thru 0x62eb57b data_size 0x1(0xdcb7 bytes) 2024-09-16T12:59:36.420 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:36.411+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:36.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout:1511 write 0x210871a thru 0x21097d1 (0x10b8 bytes) 2024-09-16T12:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout:1512 write 0x70fb467 thru 0x70fc38d (0xf27 bytes) 2024-09-16T12:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout:1513 trunc from 0x957f371 to 0xe5ca902 2024-09-16T12:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout:1514 punch from 0x25ac1f to 0x25df52, (0x3333 bytes) 2024-09-16T12:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout:1516 punch from 0x8885b90 to 0x8894dcc, (0xf23c bytes) 2024-09-16T12:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout:1517 punch from 0x72439ca to 0x7244b04, (0x113a bytes) 2024-09-16T12:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout:1519 write 0xca62694 thru 0xca704ec (0xde59 bytes) 2024-09-16T12:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout:1520 read 0x1310990 thru 0x1313c21 (0x3292 bytes) 2024-09-16T12:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout:1523 writesame 0xe2d2a3a thru 0xe2d3f95 data_size 0x2(0x155c bytes) 2024-09-16T12:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout:1524 trunc from 0xe5ca902 to 0x8853a44 2024-09-16T12:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout:1526 read 0x5cb63bd thru 0x5cc5cc4 (0xf908 bytes) 2024-09-16T12:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout:1527 trunc from 0x8853a44 to 0x9918931 2024-09-16T12:59:36.521 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:36.515+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:36.532 INFO:teuthology.orchestra.run.smithi042.stdout:1528 trunc from 0x9918931 to 0x58bedbe 2024-09-16T12:59:36.546 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:36.546 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:36.547 INFO:teuthology.orchestra.run.smithi042.stdout:1529 read 0x34a9e3f thru 0x34b9dbb (0xff7d bytes) 2024-09-16T12:59:36.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:36.571 INFO:teuthology.orchestra.run.smithi042.stdout:1530 writesame 0xcd52999 thru 0xcd57b1b data_size 0xb(0x5183 bytes) 2024-09-16T12:59:36.578 INFO:teuthology.orchestra.run.smithi042.stdout:1532 trunc from 0xcd57b1c to 0xa613bf9 2024-09-16T12:59:36.605 INFO:teuthology.orchestra.run.smithi042.stdout:1533 trunc from 0xa613bf9 to 0x4cf8d4b 2024-09-16T12:59:36.622 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:36.615+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:36.648 INFO:teuthology.orchestra.run.smithi042.stdout:1535 write 0xdfd587f thru 0xdfd61c8 (0x94a bytes) 2024-09-16T12:59:36.651 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:36.651 INFO:teuthology.orchestra.run.smithi042.stdout:1537 write 0x3f984f9 thru 0x3fa5ef3 (0xd9fb bytes) 2024-09-16T12:59:36.654 INFO:teuthology.orchestra.run.smithi042.stdout:1538 write 0x35eade4 thru 0x35eda56 (0x2c73 bytes) 2024-09-16T12:59:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:1541 read 0x686a4c7 thru 0x687639d (0xbed7 bytes) 2024-09-16T12:59:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:1543 read 0x825dfe4 thru 0x825f0cf (0x10ec bytes) 2024-09-16T12:59:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:1544 read 0x53d81bc thru 0x53dafdf (0x2e24 bytes) 2024-09-16T12:59:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:1545 write 0xd20ab34 thru 0xd21a055 (0xf522 bytes) 2024-09-16T12:59:36.664 INFO:teuthology.orchestra.run.smithi042.stdout:1546 read 0xd17fe86 thru 0xd18c7bc (0xc937 bytes) 2024-09-16T12:59:36.684 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:36.684 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.898033048999999, 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003284908, 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 10 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779657+0000", 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1239999999999999e-06 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779667+0000", 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0487e-05 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779683+0000", 2024-09-16T12:59:36.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6146000000000001e-05 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779695+0000", 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1347e-05 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779769+0000", 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.449e-05 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779918+0000", 2024-09-16T12:59:36.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000148462 2024-09-16T12:59:36.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:36.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.780088+0000", 2024-09-16T12:59:36.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017066600000000001 2024-09-16T12:59:36.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.781019+0000", 2024-09-16T12:59:36.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093024399999999999 2024-09-16T12:59:36.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782840+0000", 2024-09-16T12:59:36.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018211379999999999 2024-09-16T12:59:36.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782899+0000", 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8777000000000003e-05 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782940+0000", 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1026999999999999e-05 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:502 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 726006~2122] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.534265116, 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026083510000000001, 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 502 2024-09-16T12:59:36.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.693 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:36.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143426+0000", 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0460000000000001e-06 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143465+0000", 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143507+0000", 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1678999999999999e-05 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143523+0000", 2024-09-16T12:59:36.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6464e-05 2024-09-16T12:59:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143631+0000", 2024-09-16T12:59:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107357 2024-09-16T12:59:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143882+0000", 2024-09-16T12:59:36.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025138100000000001 2024-09-16T12:59:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.144130+0000", 2024-09-16T12:59:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024822700000000001 2024-09-16T12:59:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145527+0000", 2024-09-16T12:59:36.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001396986 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145925+0000", 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000397964 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145985+0000", 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9896999999999999e-05 2024-09-16T12:59:36.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.146031+0000", 2024-09-16T12:59:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5938e-05 2024-09-16T12:59:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.699 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:681 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [set-alloc-hint object_size 1048576 write_size 1048576,write 119408~11664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.069126979, 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035485299999999998, 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 681 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.700 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608564+0000", 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.146e-06 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608592+0000", 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8585000000000002e-05 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608635+0000", 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629000000000001e-05 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608649+0000", 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4270000000000001e-05 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608797+0000", 2024-09-16T12:59:36.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014800499999999999 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609162+0000", 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036455100000000001 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609411+0000", 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024895400000000002 2024-09-16T12:59:36.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611008+0000", 2024-09-16T12:59:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015968860000000001 2024-09-16T12:59:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611948+0000", 2024-09-16T12:59:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094050999999999998 2024-09-16T12:59:36.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612013+0000", 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5185999999999996e-05 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612109+0000", 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5808000000000002e-05 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.706 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.707 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.707 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:688 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:36.707 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:36.707 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.033797352000001, 2024-09-16T12:59:36.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027962989999999999, 2024-09-16T12:59:36.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 688 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:36.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643893+0000", 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8200000000000001e-06 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643906+0000", 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643953+0000", 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6399000000000001e-05 2024-09-16T12:59:36.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643972+0000", 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9130000000000001e-05 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644163+0000", 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019125899999999999 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644426+0000", 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026297899999999999 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644768+0000", 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000342574 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646240+0000", 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001471312 2024-09-16T12:59:36.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646617+0000", 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037757399999999999 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646658+0000", 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1083e-05 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646687+0000", 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8248000000000001e-05 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:810 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 443751~14451] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.781043448999998, 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036011279999999999, 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 810 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896648+0000", 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6109999999999998e-06 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896701+0000", 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3168999999999999e-05 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896743+0000", 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1445e-05 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896758+0000", 2024-09-16T12:59:36.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5591e-05 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896878+0000", 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120256 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897013+0000", 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134327 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897221+0000", 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020804800000000001 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.899109+0000", 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018886899999999999 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900084+0000", 2024-09-16T12:59:36.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097495899999999996 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900178+0000", 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3264999999999996e-05 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900245+0000", 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7767000000000004e-05 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.720 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:826 4.4 e38/31 4:22c6e589:::rbd_data.11b59d02cc38.000000000000005a:head v 38'8, mlcod=37'6)", 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.747524894000001, 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028983849999999999, 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930165+0000", 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.114e-06 2024-09-16T12:59:36.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930191+0000", 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6057000000000001e-05 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930222+0000", 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0959e-05 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930235+0000", 2024-09-16T12:59:36.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930349+0000", 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011372200000000001 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930402+0000", 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2766000000000003e-05 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.932954+0000", 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025517980000000001 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933061+0000", 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010770800000000001 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:829 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.735818780999999, 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024711590000000001, 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 829 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941872+0000", 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.225e-06 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941889+0000", 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6487999999999999e-05 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941926+0000", 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7727e-05 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941941+0000", 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5085999999999999e-05 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942053+0000", 2024-09-16T12:59:36.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011112600000000001 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942210+0000", 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000157653 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942387+0000", 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000176959 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.943529+0000", 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001141801 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944262+0000", 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073265199999999998 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944304+0000", 2024-09-16T12:59:36.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2200999999999998e-05 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944340+0000", 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6241000000000001e-05 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:860 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [set-alloc-hint object_size 1048576 write_size 1048576,write 997016~9314] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:36.732 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.679412321000001, 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037930170000000001, 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 860 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:36.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:36.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998280+0000", 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0969999999999999e-06 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998307+0000", 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7804999999999999e-05 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998349+0000", 2024-09-16T12:59:36.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1838000000000002e-05 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998364+0000", 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5194000000000001e-05 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998511+0000", 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014693900000000001 2024-09-16T12:59:36.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998912+0000", 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000401066 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.999376+0000", 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000463938 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001244+0000", 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018672389999999999 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001920+0000", 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067617100000000004 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002008+0000", 2024-09-16T12:59:36.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8342000000000002e-05 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002068+0000", 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0387999999999998e-05 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:995 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:36.739 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.370531228000001, 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040928859999999996, 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 995 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307159+0000", 2024-09-16T12:59:36.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9739999999999998e-06 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307174+0000", 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4134000000000001e-05 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307211+0000", 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7385000000000002e-05 2024-09-16T12:59:36.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307230+0000", 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8961e-05 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307461+0000", 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230721 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307710+0000", 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024951200000000002 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308016+0000", 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030602199999999998 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309601+0000", 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015845200000000001 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311094+0000", 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00149321 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311180+0000", 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6142000000000003e-05 2024-09-16T12:59:36.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311249+0000", 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9305000000000001e-05 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3720 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:36.746 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.782562582000001, 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024764850000000001, 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3720 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895128+0000", 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-16T12:59:36.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895141+0000", 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2824e-05 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895170+0000", 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8992e-05 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895181+0000", 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1106e-05 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895274+0000", 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3073999999999997e-05 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895417+0000", 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014300599999999999 2024-09-16T12:59:36.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895810+0000", 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039280400000000002 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897118+0000", 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00130792 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897512+0000", 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039389699999999999 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897562+0000", 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0229000000000003e-05 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897602+0000", 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9892999999999999e-05 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3724 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.766892415999999, 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002592624, 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910798+0000", 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7350000000000001e-06 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910813+0000", 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4469e-05 2024-09-16T12:59:36.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910847+0000", 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3986999999999998e-05 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910869+0000", 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2764000000000001e-05 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910939+0000", 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0086000000000006e-05 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.911119+0000", 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179598 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.911351+0000", 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023168200000000001 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.912688+0000", 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013376729999999999 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913294+0000", 2024-09-16T12:59:36.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060521299999999995 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913343+0000", 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9537999999999999e-05 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913388+0000", 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4879000000000002e-05 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4040 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.215362695, 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026268979999999999, 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4040 2024-09-16T12:59:36.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462327+0000", 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0229999999999999e-06 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462340+0000", 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3189e-05 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462373+0000", 2024-09-16T12:59:36.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2555000000000003e-05 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462386+0000", 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2935000000000001e-05 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462458+0000", 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1847999999999994e-05 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462582+0000", 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012399199999999999 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.463055+0000", 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047307700000000002 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464541+0000", 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014867649999999999 2024-09-16T12:59:36.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464841+0000", 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029975500000000002 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464912+0000", 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0344000000000003e-05 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464952+0000", 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0414999999999998e-05 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4042 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.209001391999999, 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002527208, 2024-09-16T12:59:36.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4042 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468688+0000", 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9549999999999999e-06 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468700+0000", 2024-09-16T12:59:36.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1443e-05 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468726+0000", 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6591999999999999e-05 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468737+0000", 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0951999999999999e-05 2024-09-16T12:59:36.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468841+0000", 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010335 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468995+0000", 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015392800000000001 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.469556+0000", 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00056112700000000002 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471063+0000", 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001507226 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471130+0000", 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6693999999999995e-05 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471176+0000", 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6223000000000001e-05 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471214+0000", 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7718e-05 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4044 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.203233679, 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002965217, 2024-09-16T12:59:36.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4044 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:36.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474456+0000", 2024-09-16T12:59:36.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.818e-06 2024-09-16T12:59:36.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474465+0000", 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5459999999999997e-06 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474490+0000", 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5058e-05 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474502+0000", 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1110999999999999e-05 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474610+0000", 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108606 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474815+0000", 2024-09-16T12:59:36.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020453800000000001 2024-09-16T12:59:36.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.475681+0000", 2024-09-16T12:59:36.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086620700000000002 2024-09-16T12:59:36.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477079+0000", 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001398388 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477340+0000", 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026078800000000002 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477385+0000", 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5014e-05 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477419+0000", 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4143000000000002e-05 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4046 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.495211+0000", 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.182476438, 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024109330000000001, 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4046 2024-09-16T12:59:36.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495211+0000", 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495211+0000", 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495215+0000", 2024-09-16T12:59:36.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3009999999999998e-06 2024-09-16T12:59:36.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495233+0000", 2024-09-16T12:59:36.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8804000000000002e-05 2024-09-16T12:59:36.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495275+0000", 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1634999999999997e-05 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495290+0000", 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.526e-05 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495354+0000", 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3417000000000006e-05 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495455+0000", 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010140399999999999 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.495831+0000", 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037579699999999998 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.497137+0000", 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013057310000000001 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.497549+0000", 2024-09-16T12:59:36.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000412438 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.497590+0000", 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1297000000000001e-05 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.497622+0000", 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1848999999999999e-05 2024-09-16T12:59:36.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4069 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.141858774999999, 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025697279999999999, 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4069 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535831+0000", 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8109999999999999e-06 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535841+0000", 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0267e-05 2024-09-16T12:59:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535862+0000", 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0693e-05 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535869+0000", 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7309999999999993e-06 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535958+0000", 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8721000000000007e-05 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.536094+0000", 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013532999999999999 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.536561+0000", 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000467057 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538128+0000", 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001567337 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538285+0000", 2024-09-16T12:59:36.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015667900000000001 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538355+0000", 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9991000000000001e-05 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538399+0000", 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4110999999999998e-05 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.794 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:5580 4.12 e46/40 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head v 46'11, mlcod=46'9)", 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.968693547, 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039131280000000001, 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708997+0000", 2024-09-16T12:59:36.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4109999999999999e-06 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710732+0000", 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001735002 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710766+0000", 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-05 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710779+0000", 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3137e-05 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710905+0000", 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012594799999999999 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710972+0000", 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7062000000000003e-05 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.712815+0000", 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018428419999999999 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.712907+0000", 2024-09-16T12:59:36.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2516000000000007e-05 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6800 4.1c 4:3d97daaf:::rbd_data.11b58f77e1c4.0000000000000030:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1985195~61789] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:32.580187+0000", 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0975005380000002, 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002873221, 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6800 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580187+0000", 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580187+0000", 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580190+0000", 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8880000000000001e-06 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580275+0000", 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4672000000000005e-05 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580313+0000", 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7991000000000003e-05 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580330+0000", 2024-09-16T12:59:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7425e-05 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580416+0000", 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5928999999999997e-05 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580627+0000", 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021073899999999999 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580877+0000", 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024978100000000003 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.582227+0000", 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013502729999999999 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.582955+0000", 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072785299999999999 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.583015+0000", 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0149999999999998e-05 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.583060+0000", 2024-09-16T12:59:36.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5519999999999998e-05 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6806 4.1c 4:3d97daaf:::rbd_data.11b58f77e1c4.0000000000000030:head [stat,truncate 2002817] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:32.608984+0000", 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0687033189999999, 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027044930000000001, 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6806 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.608984+0000", 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.608984+0000", 2024-09-16T12:59:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.608989+0000", 2024-09-16T12:59:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2359999999999998e-06 2024-09-16T12:59:36.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609009+0000", 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0757000000000001e-05 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609069+0000", 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0019000000000003e-05 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609094+0000", 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4119999999999999e-05 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609229+0000", 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000135773 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609396+0000", 2024-09-16T12:59:36.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000166501 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609647+0000", 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025091499999999999 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611408+0000", 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017616019999999999 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611590+0000", 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181192 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611643+0000", 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3368000000000001e-05 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611689+0000", 2024-09-16T12:59:36.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6010000000000002e-05 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7104 4.1c 4:394c1983:::rbd_data.11b58f77e1c4.0000000000000024:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.631178384, 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031870290000000001, 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7104 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046511+0000", 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9819999999999998e-06 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:36.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046518+0000", 2024-09-16T12:59:36.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4489999999999996e-06 2024-09-16T12:59:36.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:36.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046536+0000", 2024-09-16T12:59:36.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8544999999999999e-05 2024-09-16T12:59:36.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046544+0000", 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.729e-06 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046624+0000", 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9708000000000003e-05 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046755+0000", 2024-09-16T12:59:36.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130846 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049696+0000", 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029417699999999998 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:36.819 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:36.820 INFO:teuthology.orchestra.run.smithi042.stdout:1547 trunc from 0xdfd61c9 to 0xd88d71 2024-09-16T12:59:36.821 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:36.823 INFO:teuthology.orchestra.run.smithi042.stdout:1549 read 0xb69bc3 thru 0xb74bbb (0xaff9 bytes) 2024-09-16T12:59:36.823 INFO:teuthology.orchestra.run.smithi042.stdout:1550 punch from 0x619a6f to 0x626a0b, (0xcf9c bytes) 2024-09-16T12:59:36.823 INFO:teuthology.orchestra.run.smithi042.stdout:1552 write 0xea43956 thru 0xea489ac (0x5057 bytes) 2024-09-16T12:59:36.823 INFO:teuthology.orchestra.run.smithi042.stdout:1553 writesame 0xbbf9090 thru 0xbc0368d data_size 0x1(0xa5fe bytes) 2024-09-16T12:59:36.823 INFO:teuthology.orchestra.run.smithi042.stdout:1554 read 0x968d4e6 thru 0x9699d21 (0xc83c bytes) 2024-09-16T12:59:36.823 INFO:teuthology.orchestra.run.smithi042.stdout:1555 trunc from 0xea489ad to 0xa20549f 2024-09-16T12:59:36.823 INFO:teuthology.orchestra.run.smithi042.stdout:1556 writesame 0xebc07a7 thru 0xebd055f data_size 0x15(0xfdb9 bytes) 2024-09-16T12:59:36.823 INFO:teuthology.orchestra.run.smithi042.stdout:1557 writesame 0x5cf518b thru 0x5cfd92b data_size 0x1(0x87a1 bytes) 2024-09-16T12:59:36.824 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:36.811+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:36.848 INFO:teuthology.orchestra.run.smithi042.stdout:1558 writesame 0x3088e47 thru 0x30957b5 data_size 0x1(0xc96f bytes) 2024-09-16T12:59:36.888 INFO:teuthology.orchestra.run.smithi042.stdout:1560 writesame 0x24226ef thru 0x242c8be data_size 0x18(0xa1d0 bytes) 2024-09-16T12:59:36.892 INFO:teuthology.orchestra.run.smithi042.stdout:1561 punch from 0x96513f9 to 0x9659c00, (0x8807 bytes) 2024-09-16T12:59:36.893 INFO:teuthology.orchestra.run.smithi042.stdout:1562 write 0xc51b62c thru 0xc526a07 (0xb3dc bytes) 2024-09-16T12:59:36.896 INFO:teuthology.orchestra.run.smithi042.stdout:1564 write 0x1511836 thru 0x151b297 (0x9a62 bytes) 2024-09-16T12:59:36.898 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:36.899 INFO:teuthology.orchestra.run.smithi042.stdout:1566 read 0xc707915 thru 0xc714337 (0xca23 bytes) 2024-09-16T12:59:36.899 INFO:teuthology.orchestra.run.smithi042.stdout:1567 punch from 0x36ce384 to 0x36d7e7c, (0x9af8 bytes) 2024-09-16T12:59:36.922 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:36.911+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:36.927 INFO:teuthology.orchestra.run.smithi042.stdout:1568 trunc from 0xebd0560 to 0x136213d 2024-09-16T12:59:36.962 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T12:59:36.963 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:36.964 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:36.964 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:36.964 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:36.970 INFO:teuthology.orchestra.run.smithi042.stdout:1569 writesame 0xac91f07 thru 0xac9d6cd data_size 0x5b(0xb7c7 bytes) 2024-09-16T12:59:36.973 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:36.973 INFO:teuthology.orchestra.run.smithi042.stdout:1571 read 0x8a1c63c thru 0x8a22ec8 (0x688d bytes) 2024-09-16T12:59:36.974 INFO:teuthology.orchestra.run.smithi042.stdout:1572 write 0x92ae4d4 thru 0x92b54ff (0x702c bytes) 2024-09-16T12:59:36.988 INFO:teuthology.orchestra.run.smithi042.stdout:1573 trunc from 0xac9d6ce to 0x320cb53 2024-09-16T12:59:37.003 INFO:teuthology.orchestra.run.smithi042.stdout:1574 read 0x83abd4 thru 0x83d02b (0x2458 bytes) 2024-09-16T12:59:37.004 INFO:teuthology.orchestra.run.smithi042.stdout:1575 punch from 0x869a08 to 0x874da3, (0xb39b bytes) 2024-09-16T12:59:37.004 INFO:teuthology.orchestra.run.smithi042.stdout:1577 read 0x20cf1a3 thru 0x20d9170 (0x9fce bytes) 2024-09-16T12:59:37.010 INFO:teuthology.orchestra.run.smithi042.stdout:1579 trunc from 0x320cb53 to 0x1610f 2024-09-16T12:59:37.022 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:37.015+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout:1581 write 0x91be858 thru 0x91cc58b (0xdd34 bytes) 2024-09-16T12:59:37.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout:1582 punch from 0x3475cf to 0x352e67, (0xb898 bytes) 2024-09-16T12:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout:1583 read 0x5fc80ba thru 0x5fd3048 (0xaf8f bytes) 2024-09-16T12:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout:1584 writesame 0x9613c thru 0x9e5d0 data_size 0x1(0x8495 bytes) 2024-09-16T12:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout:1585 trunc from 0x91cc58c to 0x730df5f 2024-09-16T12:59:37.123 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:37.115+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:37.132 INFO:teuthology.orchestra.run.smithi042.stdout:1586 punch from 0x2a51a82 to 0x2a5a6ed, (0x8c6b bytes) 2024-09-16T12:59:37.153 INFO:teuthology.orchestra.run.smithi042.stdout:1587 write 0xcf2e478 thru 0xcf30c12 (0x279b bytes) 2024-09-16T12:59:37.161 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:37.161 INFO:teuthology.orchestra.run.smithi042.stdout:1589 read 0xabc2193 thru 0xabc644b (0x42b9 bytes) 2024-09-16T12:59:37.161 INFO:teuthology.orchestra.run.smithi042.stdout:1590 write 0x8e99aba thru 0x8e9a213 (0x75a bytes) 2024-09-16T12:59:37.167 INFO:teuthology.orchestra.run.smithi042.stdout:1591 read 0x65e894b thru 0x65f1f16 (0x95cc bytes) 2024-09-16T12:59:37.167 INFO:teuthology.orchestra.run.smithi042.stdout:1593 read 0x1b7becf thru 0x1b7cf42 (0x1074 bytes) 2024-09-16T12:59:37.168 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:37.168 INFO:teuthology.orchestra.run.smithi042.stdout:1595 writesame 0x561c8de thru 0x561de89 data_size 0x13(0x15ac bytes) 2024-09-16T12:59:37.174 INFO:teuthology.orchestra.run.smithi042.stdout:1596 punch from 0xa4cf535 to 0xa4d5c4a, (0x6715 bytes) 2024-09-16T12:59:37.174 INFO:teuthology.orchestra.run.smithi042.stdout:1597 write 0xa644da6 thru 0xa6518a5 (0xcb00 bytes) 2024-09-16T12:59:37.178 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:37.178 INFO:teuthology.orchestra.run.smithi042.stdout:1599 read 0xc88e557 thru 0xc898c91 (0xa73b bytes) 2024-09-16T12:59:37.179 INFO:teuthology.orchestra.run.smithi042.stdout:1600 read 0x6d3d9b4 thru 0x6d49d25 (0xc372 bytes) 2024-09-16T12:59:37.179 INFO:teuthology.orchestra.run.smithi042.stdout:1601 read 0x3df0c61 thru 0x3df46fc (0x3a9c bytes) 2024-09-16T12:59:37.182 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:37.182 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:37.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:37.182 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:37.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.182 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:575 4.17 4:eab06920:::rbd_data.11b59d02cc38.0000000000000075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:37.182 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:37.182 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.862968158000001, 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049089249999999997, 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 575 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312345+0000", 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.353e-06 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312352+0000", 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7499999999999997e-06 2024-09-16T12:59:37.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312370+0000", 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8210999999999999e-05 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312391+0000", 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0968e-05 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312965+0000", 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057344399999999999 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313134+0000", 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016979199999999999 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313322+0000", 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018742300000000001 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314160+0000", 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083821799999999995 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317190+0000", 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030301339999999999 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317219+0000", 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8368999999999999e-05 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317253+0000", 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4263e-05 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:950 4.17 4:e9df2e13:::rbd_data.11b59d02cc38.0000000000000007:head [write 983040~36251] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.202396+0000", 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.972916445999999, 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043336290000000003, 2024-09-16T12:59:37.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 950 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202396+0000", 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202396+0000", 2024-09-16T12:59:37.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202397+0000", 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7069999999999999e-06 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202433+0000", 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5688000000000001e-05 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202452+0000", 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8550999999999999e-05 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202472+0000", 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0565999999999999e-05 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202586+0000", 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113328 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202802+0000", 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021687200000000001 2024-09-16T12:59:37.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.203171+0000", 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036829700000000001 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.204781+0000", 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016104769999999999 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.206481+0000", 2024-09-16T12:59:37.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017001989999999999 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.206620+0000", 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139004 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.206729+0000", 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010894 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.195 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:985 4.11 4:882565b9:::rbd_data.11b59d02cc38.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.304439+0000", 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.870873389, 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042693189999999997, 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 985 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304439+0000", 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304439+0000", 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304443+0000", 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-06 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304464+0000", 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0767999999999998e-05 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304526+0000", 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1865999999999993e-05 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304575+0000", 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9178999999999998e-05 2024-09-16T12:59:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304715+0000", 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014015699999999999 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304920+0000", 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020457299999999999 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.305215+0000", 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029488399999999998 2024-09-16T12:59:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307142+0000", 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019277599999999999 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308529+0000", 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001386424 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308632+0000", 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010343700000000001 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308708+0000", 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5871000000000006e-05 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1001 4.1d 4:b8dd3499:::rbd_data.11b59d02cc38.0000000000000051:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.309177+0000", 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.866135194999998, 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049686330000000001, 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1001 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309177+0000", 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309177+0000", 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309179+0000", 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1320000000000001e-06 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309187+0000", 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2479999999999996e-06 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309216+0000", 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8884e-05 2024-09-16T12:59:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309228+0000", 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1992e-05 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309365+0000", 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013697399999999999 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309849+0000", 2024-09-16T12:59:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048348 2024-09-16T12:59:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309985+0000", 2024-09-16T12:59:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013675300000000001 2024-09-16T12:59:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311601+0000", 2024-09-16T12:59:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001615472 2024-09-16T12:59:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.314014+0000", 2024-09-16T12:59:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024127950000000001 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.314102+0000", 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8324000000000003e-05 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.314146+0000", 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3578999999999997e-05 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1145 4.17 4:edd7f3ed:::image_client.0-clone2.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.733224+0000", 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.442087787999998, 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053216499999999998, 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1145 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733224+0000", 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733224+0000", 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733226+0000", 2024-09-16T12:59:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0109999999999999e-06 2024-09-16T12:59:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733234+0000", 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3440000000000003e-06 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733255+0000", 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1347000000000001e-05 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733266+0000", 2024-09-16T12:59:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.117e-05 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733330+0000", 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3577000000000005e-05 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733339+0000", 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0520000000000007e-06 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.738402+0000", 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050630680000000004 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.738546+0000", 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014408100000000001 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1148 4.0 4:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.741509+0000", 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.433803266999998, 2024-09-16T12:59:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055772950000000003, 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1148 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741509+0000", 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741509+0000", 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741511+0000", 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8029999999999999e-06 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741519+0000", 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0779999999999996e-06 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741541+0000", 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2609999999999999e-05 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741552+0000", 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1001e-05 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741622+0000", 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.004e-05 2024-09-16T12:59:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741786+0000", 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000163251 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741966+0000", 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018033400000000001 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.742883+0000", 2024-09-16T12:59:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091699900000000005 2024-09-16T12:59:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.746880+0000", 2024-09-16T12:59:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039970769999999999 2024-09-16T12:59:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.747012+0000", 2024-09-16T12:59:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000131698 2024-09-16T12:59:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.747086+0000", 2024-09-16T12:59:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4404000000000001e-05 2024-09-16T12:59:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:3502 4.1b e41/31 4:dbfa2cf4:::rbd_data.11b5c378985b.0000000000000013:head v 41'8, mlcod=38'6)", 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:21.969284+0000", 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.206028278, 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060809890000000002, 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.969284+0000", 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.969284+0000", 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.969286+0000", 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2079999999999999e-06 2024-09-16T12:59:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971841+0000", 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025551620000000001 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971871+0000", 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0230999999999999e-05 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971883+0000", 2024-09-16T12:59:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1046000000000001e-05 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971961+0000", 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8560999999999995e-05 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.972001+0000", 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0204999999999999e-05 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.975320+0000", 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033191800000000001 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.975365+0000", 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4396e-05 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3536 4.14 4:2a646c7f:::rbd_data.11b5c378985b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.025341+0000", 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.149971063000001, 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067260820000000004, 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3536 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025341+0000", 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025341+0000", 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025343+0000", 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.739e-06 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025571+0000", 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022837000000000001 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025590+0000", 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8360999999999999e-05 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025597+0000", 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1910000000000003e-06 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025667+0000", 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0723999999999996e-05 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025848+0000", 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000180679 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.027038+0000", 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001189854 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.031540+0000", 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045024829999999998 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.031922+0000", 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000381689 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.032005+0000", 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3141999999999998e-05 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.032067+0000", 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1849999999999999e-05 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6872 4.17 4:eebd2e7d:::rbd_data.11b58f77e1c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:32.718189+0000", 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.4571232329999999, 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051813229999999998, 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6872 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718189+0000", 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718189+0000", 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718191+0000", 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9400000000000001e-06 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718200+0000", 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0890000000000006e-06 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718225+0000", 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4746999999999999e-05 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718235+0000", 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9760000000000007e-06 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718302+0000", 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7163999999999996e-05 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718489+0000", 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000187677 2024-09-16T12:59:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.719451+0000", 2024-09-16T12:59:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096120600000000004 2024-09-16T12:59:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.723228+0000", 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003777772 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.723267+0000", 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8374000000000002e-05 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.723328+0000", 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0778e-05 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.723370+0000", 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2599999999999999e-05 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7072 4.17 4:ee8c78e5:::rbd_object_map.11b58f77e1c4.000000000000000c:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.038229+0000", 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1370827610000003, 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053143549999999998, 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7072 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038229+0000", 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038229+0000", 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038231+0000", 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7320000000000001e-06 2024-09-16T12:59:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038239+0000", 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0199999999999994e-06 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038259+0000", 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0228e-05 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038268+0000", 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8859999999999993e-06 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038336+0000", 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7878000000000003e-05 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038492+0000", 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000156203 2024-09-16T12:59:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039076+0000", 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058399299999999999 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.041475+0000", 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023988569999999999 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043381+0000", 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019060119999999999 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043482+0000", 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100463 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043544+0000", 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2082999999999995e-05 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7075 4.19 4:9a86cce4:::rbd_data.11b58f77e1c4.000000000000001e:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.039147+0000", 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1361654809999999, 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045161819999999997, 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7075 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039147+0000", 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039147+0000", 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039148+0000", 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0210000000000001e-06 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039153+0000", 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3449999999999996e-06 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039170+0000", 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7011e-05 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039177+0000", 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8229999999999997e-06 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039255+0000", 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8360000000000002e-05 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039578+0000", 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032231700000000002 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043663+0000", 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004085305 2024-09-16T12:59:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7076 4.17 4:ee8c78e5:::rbd_object_map.11b58f77e1c4.000000000000000c:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.039494+0000", 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1358184309999997, 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055361380000000003, 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7076 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039494+0000", 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039494+0000", 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039495+0000", 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.569999999999999e-07 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039500+0000", 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5729999999999998e-06 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039513+0000", 2024-09-16T12:59:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2505e-05 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039519+0000", 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9850000000000003e-06 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039814+0000", 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000294803 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039987+0000", 2024-09-16T12:59:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017362200000000001 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040675+0000", 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00068771100000000003 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043937+0000", 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032622720000000001 2024-09-16T12:59:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044428+0000", 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049094700000000004 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044554+0000", 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012569400000000001 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045030+0000", 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047606900000000003 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7078 4.17 4:ee8c78e5:::rbd_object_map.11b58f77e1c4.000000000000000c:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.040912+0000", 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.134400501, 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044986549999999998, 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7078 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040912+0000", 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040912+0000", 2024-09-16T12:59:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040913+0000", 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1629999999999999e-06 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040919+0000", 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7610000000000004e-06 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040932+0000", 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040939+0000", 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5729999999999996e-06 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.041520+0000", 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058121099999999999 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.041687+0000", 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000167694 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.042457+0000", 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076913899999999996 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044193+0000", 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017368419999999999 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045063+0000", 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086991599999999998 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045349+0000", 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028614200000000003 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045410+0000", 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0800000000000001e-05 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7095 4.17 4:ef19ffb6:::rbd_data.11b58f77e1c4.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.044709+0000", 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1306031320000001, 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005134502, 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7095 2024-09-16T12:59:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044709+0000", 2024-09-16T12:59:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044709+0000", 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044710+0000", 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2240000000000001e-06 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044728+0000", 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7884e-05 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044753+0000", 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4905e-05 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044764+0000", 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0841e-05 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045485+0000", 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072079700000000002 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045650+0000", 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016573 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046549+0000", 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000899024 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049015+0000", 2024-09-16T12:59:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024659930000000001 2024-09-16T12:59:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049745+0000", 2024-09-16T12:59:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072982600000000004 2024-09-16T12:59:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049806+0000", 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1106999999999996e-05 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049844+0000", 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7171e-05 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7096 4.17 4:ef19ffb6:::rbd_data.11b58f77e1c4.0000000000000018:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.044779+0000", 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1305332640000003, 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061860320000000002, 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7096 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044779+0000", 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044779+0000", 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044780+0000", 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1100000000000005e-07 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044784+0000", 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.42e-06 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044795+0000", 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0431000000000001e-05 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044799+0000", 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.797e-06 2024-09-16T12:59:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.047738+0000", 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002938689 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.047939+0000", 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020134600000000001 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.048152+0000", 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021225000000000001 2024-09-16T12:59:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.050856+0000", 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027047529999999998 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.050901+0000", 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4799000000000003e-05 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.050944+0000", 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2574000000000003e-05 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.050965+0000", 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1162000000000002e-05 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8149 4.f 4:f373df67:::rbd_header.11b59378797:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:35.940128+0000", 2024-09-16T12:59:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.2351838390000001, 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084867789999999999, 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8149 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940128+0000", 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940128+0000", 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940131+0000", 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4389999999999999e-06 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940141+0000", 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0540000000000001e-05 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940164+0000", 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2872000000000001e-05 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940176+0000", 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1462e-05 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940245+0000", 2024-09-16T12:59:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9686999999999999e-05 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940253+0000", 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0250000000000002e-06 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.944461+0000", 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042077379999999999 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.944583+0000", 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121668 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.944970+0000", 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038757599999999999 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.945593+0000", 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062260699999999996 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.948452+0000", 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002858817 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.948547+0000", 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5495000000000006e-05 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.948615+0000", 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7853000000000002e-05 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8579 4.11 4:895e0e12:::rbd_data.11b55eb65e78.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.742439+0000", 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.43287276600000002, 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025774687000000001, 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8579 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742439+0000", 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742439+0000", 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742443+0000", 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.473e-06 2024-09-16T12:59:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742461+0000", 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8329999999999999e-05 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742504+0000", 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3098e-05 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742520+0000", 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5982999999999999e-05 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742635+0000", 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000114537 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742782+0000", 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014705100000000001 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.749390+0000", 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066079370000000004 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.767807+0000", 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018417086999999999 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.767925+0000", 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011807300000000001 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.768098+0000", 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172876 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.768214+0000", 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116242 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8611 4.11 4:895e0e12:::rbd_data.11b55eb65e78.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.815402+0000", 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.35991023799999999, 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927149, 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8611 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815402+0000", 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815402+0000", 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815405+0000", 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8030000000000001e-06 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815416+0000", 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1741e-05 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815463+0000", 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6303e-05 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815474+0000", 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1189000000000001e-05 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815580+0000", 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010592899999999999 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815765+0000", 2024-09-16T12:59:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185095 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.821526+0000", 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057609820000000004 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845015+0000", 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023488841999999999 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845048+0000", 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2913999999999997e-05 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845204+0000", 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015631600000000001 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845329+0000", 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012503500000000001 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:8611 4.11 e54/31 ondisk, result = 0)", 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.839266+0000", 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.33604621499999998, 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057569609999999997, 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839266+0000", 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839266+0000", 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839267+0000", 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7400000000000002e-07 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839275+0000", 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1699999999999997e-06 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839290+0000", 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5107e-05 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839296+0000", 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4320000000000004e-06 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.844965+0000", 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056682069999999998 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.844990+0000", 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.516e-05 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845023+0000", 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3011e-05 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8622 4.11 4:895e0e12:::rbd_data.11b55eb65e78.0000000000000005:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.925704+0000", 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.24960786600000001, 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040638050000000002, 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8622 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.925704+0000", 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.925704+0000", 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.925708+0000", 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6629999999999999e-06 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.925724+0000", 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6484e-05 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.925780+0000", 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5874000000000001e-05 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.925801+0000", 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0256999999999999e-05 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.925894+0000", 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3336000000000002e-05 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.925993+0000", 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9315999999999996e-05 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.926141+0000", 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014743199999999999 2024-09-16T12:59:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.926953+0000", 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00081242900000000004 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.929653+0000", 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026997499999999999 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.929720+0000", 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7422999999999995e-05 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.929768+0000", 2024-09-16T12:59:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840999999999998e-05 2024-09-16T12:59:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:37.300 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:37.302 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:37.303 INFO:teuthology.orchestra.run.smithi042.stdout:1604 write 0xdac7777 thru 0xdacc43d (0x4cc7 bytes) 2024-09-16T12:59:37.303 INFO:teuthology.orchestra.run.smithi042.stdout:1605 writesame 0x2e1b1ad thru 0x2e20830 data_size 0x62(0x5684 bytes) 2024-09-16T12:59:37.303 INFO:teuthology.orchestra.run.smithi042.stdout:1608 read 0x1482718 thru 0x1484bc8 (0x24b1 bytes) 2024-09-16T12:59:37.303 INFO:teuthology.orchestra.run.smithi042.stdout:1609 read 0x4a1ed2b thru 0x4a27356 (0x862c bytes) 2024-09-16T12:59:37.303 INFO:teuthology.orchestra.run.smithi042.stdout:1610 write 0xb93c812 thru 0xb944df5 (0x85e4 bytes) 2024-09-16T12:59:37.303 INFO:teuthology.orchestra.run.smithi042.stdout:1611 read 0x2c5da1b thru 0x2c63985 (0x5f6b bytes) 2024-09-16T12:59:37.303 INFO:teuthology.orchestra.run.smithi042.stdout:1612 clone 11 order 24 su 1048576 sc 7 2024-09-16T12:59:37.303 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:37.295+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:37.402 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:37.399+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:37.409 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:37.409 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:37.410 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:37.410 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:37.419 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:37.502 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:37.495+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:37.520 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:37.521 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:37.530 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:37.603 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:37.595+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:37.632 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:590 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.287602699000001, 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034258392999999998, 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 590 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339271+0000", 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339281+0000", 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6670000000000008e-06 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339307+0000", 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5907000000000001e-05 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339317+0000", 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0356999999999999e-05 2024-09-16T12:59:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339412+0000", 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4248999999999998e-05 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339533+0000", 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012184900000000001 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.347489+0000", 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079559599999999998 2024-09-16T12:59:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373423+0000", 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025933971 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373435+0000", 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1924e-05 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373492+0000", 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6842999999999998e-05 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373528+0000", 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1298 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.076549+0000", 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.550323459000001, 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012581233000000001, 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1298 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076549+0000", 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076549+0000", 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076550+0000", 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9599999999999998e-07 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076554+0000", 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8289999999999998e-06 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076565+0000", 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0446e-05 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076569+0000", 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5220000000000001e-06 2024-09-16T12:59:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.082476+0000", 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059068799999999998 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.082603+0000", 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012666399999999999 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083102+0000", 2024-09-16T12:59:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049894900000000005 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.087350+0000", 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042485379999999996 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.088968+0000", 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001617804 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089056+0000", 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7690999999999998e-05 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089130+0000", 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4014e-05 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5063 4.13 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22711298~51120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.317457900999999, 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111447293, 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5063 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309416+0000", 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358861+0000", 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049444717999999999 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358893+0000", 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2279e-05 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358902+0000", 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4390000000000006e-06 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359062+0000", 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016014100000000001 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359330+0000", 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026734699999999997 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.362566+0000", 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032358769999999999 2024-09-16T12:59:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.417541+0000", 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054975345000000002 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420706+0000", 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031651380000000001 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420790+0000", 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4283999999999995e-05 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420862+0000", 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1138999999999998e-05 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5225 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.804570+0000", 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.8223025810000006, 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027003329999999999, 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5225 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804571+0000", 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6309999999999999e-06 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804579+0000", 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804600+0000", 2024-09-16T12:59:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1114000000000001e-05 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804613+0000", 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3040000000000001e-05 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804713+0000", 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9854999999999999e-05 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804844+0000", 2024-09-16T12:59:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130963 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.810955+0000", 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061105880000000001 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831495+0000", 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020540467 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831507+0000", 2024-09-16T12:59:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831544+0000", 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7413000000000002e-05 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831573+0000", 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8836e-05 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5504 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.468273+0000", 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1585989750000003, 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020881448, 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5504 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T12:59:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468276+0000", 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8370000000000001e-06 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468290+0000", 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.383e-05 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468327+0000", 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468341+0000", 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.331e-05 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468439+0000", 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8746999999999995e-05 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468593+0000", 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015343999999999999 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.472806+0000", 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042129849999999998 2024-09-16T12:59:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488823+0000", 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016017686999999999 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488936+0000", 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112074 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489029+0000", 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3801000000000005e-05 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489155+0000", 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125248 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8352 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.280585+0000", 2024-09-16T12:59:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.346287185, 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014415468000000001, 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8352 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280585+0000", 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280585+0000", 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280587+0000", 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0229999999999999e-06 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280597+0000", 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6139999999999998e-06 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280617+0000", 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9969000000000001e-05 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280626+0000", 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.290184+0000", 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009558024 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.290251+0000", 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6210000000000005e-05 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.290656+0000", 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040517800000000002 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294871+0000", 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042150529999999999 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294885+0000", 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4154999999999999e-05 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294956+0000", 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1101999999999998e-05 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295000+0000", 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4339999999999999e-05 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8353 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.280701+0000", 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3461710140000001, 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015810297000000001, 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8353 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280701+0000", 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280701+0000", 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280702+0000", 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5000000000000001e-07 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280708+0000", 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6130000000000003e-06 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280730+0000", 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.211e-05 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280737+0000", 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.323e-06 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.291663+0000", 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010926282000000001 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.291729+0000", 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5908000000000007e-05 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.291994+0000", 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026483799999999999 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296145+0000", 2024-09-16T12:59:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004150969 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296448+0000", 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030309 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296484+0000", 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5488999999999999e-05 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296511+0000", 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7725e-05 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8354 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.281367+0000", 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3455049049999999, 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016884742000000001, 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8354 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281370+0000", 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3159999999999999e-06 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281382+0000", 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281413+0000", 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1519000000000001e-05 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281435+0000", 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1812999999999999e-05 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292521+0000", 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011085915999999999 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292628+0000", 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107311 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293108+0000", 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047976600000000001 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297972+0000", 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048640200000000001 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298174+0000", 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020222 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298226+0000", 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2058000000000002e-05 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298252+0000", 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5743000000000001e-05 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8355 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.282597+0000", 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3442750059999999, 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015824709999999999, 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8355 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282597+0000", 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282597+0000", 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282600+0000", 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3870000000000002e-06 2024-09-16T12:59:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282613+0000", 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3733e-05 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282634+0000", 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0766000000000001e-05 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282645+0000", 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0783e-05 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293789+0000", 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011144664 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293857+0000", 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7267999999999994e-05 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294144+0000", 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000287025 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297979+0000", 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003834811 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298314+0000", 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000335092 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298371+0000", 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7046999999999998e-05 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298422+0000", 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1134000000000002e-05 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8357 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.285472+0000", 2024-09-16T12:59:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3414001899999999, 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015109900000000001, 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8357 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285472+0000", 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285472+0000", 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285474+0000", 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9039999999999999e-06 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285481+0000", 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3640000000000001e-06 2024-09-16T12:59:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285495+0000", 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3963e-05 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285503+0000", 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.712e-06 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295041+0000", 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095385009999999996 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295111+0000", 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0097000000000003e-05 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295515+0000", 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040313399999999998 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297985+0000", 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024699729999999999 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.300526+0000", 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025414029999999998 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.300560+0000", 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4e-05 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.300582+0000", 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1849e-05 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8580 4.e 4:746c7c9d:::rbd_data.11b55eb65e78.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.744435+0000", 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.88243685900000002, 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012615856, 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8580 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744435+0000", 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744435+0000", 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744437+0000", 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1830000000000001e-06 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744449+0000", 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1800000000000001e-05 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744485+0000", 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5918999999999999e-05 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744503+0000", 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7759999999999999e-05 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744612+0000", 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109368 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744744+0000", 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131509 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.747374+0000", 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002629981 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.756812+0000", 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094380720000000005 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.756836+0000", 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3955e-05 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.756930+0000", 2024-09-16T12:59:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4561999999999993e-05 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.757051+0000", 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120747 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8674 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009917+0000", 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.61695490500000005, 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01501186, 2024-09-16T12:59:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8674 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009917+0000", 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009917+0000", 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009918+0000", 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009920+0000", 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.379e-06 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009924+0000", 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5249999999999999e-06 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009928+0000", 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1200000000000002e-06 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017969+0000", 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080410150000000003 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.018191+0000", 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022191999999999999 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.019035+0000", 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084407199999999996 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.022090+0000", 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030553849999999999 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024810+0000", 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027204009999999999 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024878+0000", 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7704000000000001e-05 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024929+0000", 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0983000000000003e-05 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8675 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009930+0000", 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.61694173500000005, 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015359810999999999, 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8675 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009930+0000", 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009930+0000", 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009931+0000", 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6100000000000002e-07 2024-09-16T12:59:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009934+0000", 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8559999999999998e-06 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009938+0000", 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5369999999999999e-06 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009942+0000", 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8970000000000001e-06 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020310+0000", 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010367833 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020475+0000", 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016531400000000001 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.021011+0000", 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053554600000000003 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025005+0000", 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039943360000000002 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025130+0000", 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125165 2024-09-16T12:59:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025226+0000", 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5753000000000003e-05 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025290+0000", 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4213000000000003e-05 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8676 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.61692723000000005, 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015451705, 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8676 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009948+0000", 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6249999999999999e-06 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009952+0000", 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3449999999999999e-06 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009955+0000", 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0510000000000001e-06 2024-09-16T12:59:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.022130+0000", 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012175002000000001 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.022734+0000", 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060358299999999998 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023097+0000", 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036315300000000001 2024-09-16T12:59:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025115+0000", 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020180720000000001 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025305+0000", 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018960299999999999 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025342+0000", 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7684e-05 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025397+0000", 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4231000000000003e-05 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8677 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009958+0000", 2024-09-16T12:59:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.61691393500000002, 2024-09-16T12:59:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01642155, 2024-09-16T12:59:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8677 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T12:59:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009959+0000", 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2899999999999999e-07 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009961+0000", 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3760000000000002e-06 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009965+0000", 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009968+0000", 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1190000000000001e-06 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023570+0000", 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013601317999999999 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023695+0000", 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012563400000000001 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024191+0000", 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000495434 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026285+0000", 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020948360000000001 2024-09-16T12:59:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026295+0000", 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8889999999999997e-06 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026338+0000", 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2253999999999999e-05 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026380+0000", 2024-09-16T12:59:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2153000000000001e-05 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8678 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009972+0000", 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.61689968299999998, 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020138508999999999, 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8678 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009973+0000", 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3000000000000002e-07 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009975+0000", 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4739999999999999e-06 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009980+0000", 2024-09-16T12:59:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4399999999999998e-06 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009983+0000", 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1209999999999998e-06 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025464+0000", 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015480918 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025536+0000", 2024-09-16T12:59:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2212000000000006e-05 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025768+0000", 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000232186 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029964+0000", 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041957899999999996 2024-09-16T12:59:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029981+0000", 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6634e-05 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030070+0000", 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9487999999999994e-05 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030111+0000", 2024-09-16T12:59:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0915999999999999e-05 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8681 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.017890+0000", 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.60898249900000001, 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013352773, 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8681 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017890+0000", 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017890+0000", 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017892+0000", 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1830000000000001e-06 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017905+0000", 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2782000000000001e-05 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017928+0000", 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3828e-05 2024-09-16T12:59:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017940+0000", 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1756999999999999e-05 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026416+0000", 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084759359999999999 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026489+0000", 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2618000000000002e-05 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026837+0000", 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000348389 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030134+0000", 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032973630000000002 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031172+0000", 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001037846 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031212+0000", 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9242999999999997e-05 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031242+0000", 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0827999999999997e-05 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8682 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.020416+0000", 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.60645586500000004, 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013951481, 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8682 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020416+0000", 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020416+0000", 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020418+0000", 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6270000000000001e-06 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020427+0000", 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5380000000000008e-06 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020444+0000", 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6770999999999999e-05 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020455+0000", 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1088e-05 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.027337+0000", 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068815229999999996 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.027426+0000", 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8727000000000006e-05 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.027954+0000", 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052817300000000005 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.033199+0000", 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052455879999999998 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.034274+0000", 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010743759999999999 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.034311+0000", 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7314999999999998e-05 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.034368+0000", 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6755000000000001e-05 2024-09-16T12:59:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8686 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.023953+0000", 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.60291933399999997, 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012887233, 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8686 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023953+0000", 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023953+0000", 2024-09-16T12:59:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023954+0000", 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1179999999999999e-06 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023960+0000", 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3740000000000003e-06 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023971+0000", 2024-09-16T12:59:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0274e-05 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023977+0000", 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.353e-06 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030172+0000", 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061950809999999999 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030237+0000", 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4844999999999993e-05 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030529+0000", 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029214699999999998 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.035705+0000", 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051761699999999999 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.036741+0000", 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010355480000000001 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.036796+0000", 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5460000000000001e-05 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.036840+0000", 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3863000000000003e-05 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8718 4.1b 4:dbb4ad27:::rbd_data.11b55eb65e78.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.086287+0000", 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.54058472199999996, 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028260378999999999, 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8718 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086290+0000", 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1950000000000002e-06 2024-09-16T12:59:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086300+0000", 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0458999999999999e-05 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086331+0000", 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1117e-05 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086344+0000", 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3191e-05 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086423+0000", 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8288999999999995e-05 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086530+0000", 2024-09-16T12:59:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010720799999999999 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.091612+0000", 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050817429999999997 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114274+0000", 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022662832000000001 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114386+0000", 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011155899999999999 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114475+0000", 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9043999999999997e-05 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114548+0000", 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2742000000000003e-05 2024-09-16T12:59:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:37.749 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:37.751 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:37.751 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:37.747+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:37.851 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:37.847+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:37.904 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:37.904 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:37.919 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:37.952 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:37.947+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:37.972 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T12:59:38.020 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:38.024 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:38.024 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:38.033 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:38.078 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:38.075+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:38.084 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone10 from 0xdacc43e (overlap 0x1610f) to 0x0 2024-09-16T12:59:38.154 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:38.147+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:38.167 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:38.176 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-16T12:59:38.181 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T12:59:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.547252704000002, 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006297965, 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629315+0000", 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7080000000000002e-06 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629324+0000", 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0250000000000008e-06 2024-09-16T12:59:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629340+0000", 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.59e-05 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629351+0000", 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1236999999999999e-05 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629565+0000", 2024-09-16T12:59:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000213482 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629582+0000", 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7482999999999998e-05 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.631980+0000", 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023977830000000001 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.632247+0000", 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000266867 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.632356+0000", 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010894 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.633744+0000", 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013876240000000001 2024-09-16T12:59:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635507+0000", 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001763288 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635562+0000", 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5096999999999998e-05 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635608+0000", 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5531000000000002e-05 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.535187106999999, 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063927960000000001, 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641378+0000", 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6429999999999999e-06 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641390+0000", 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1912e-05 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641422+0000", 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2666999999999997e-05 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641448+0000", 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5636e-05 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641623+0000", 2024-09-16T12:59:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017479299999999999 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641641+0000", 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8369999999999999e-05 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.644629+0000", 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00298745 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.644995+0000", 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000366688 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.645348+0000", 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035283200000000003 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647159+0000", 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018104760000000001 2024-09-16T12:59:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647631+0000", 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047253799999999999 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647708+0000", 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6715e-05 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647768+0000", 2024-09-16T12:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0075999999999999e-05 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.527607148000001, 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006003988, 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 7 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648958+0000", 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0280000000000001e-06 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648970+0000", 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1374e-05 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649002+0000", 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2298999999999997e-05 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649014+0000", 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2442e-05 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649099+0000", 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4655999999999997e-05 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649107+0000", 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0800000000000006e-06 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.652487+0000", 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003380241 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.652733+0000", 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024601000000000001 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.653091+0000", 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035755599999999998 2024-09-16T12:59:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654603+0000", 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001512163 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654795+0000", 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000192179 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654875+0000", 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0211999999999997e-05 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654959+0000", 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3747999999999998e-05 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:513 4.8 4:132f6b09:::rbd_data.11b59d02cc38.000000000000008c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.995469787000001, 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054902010000000001, 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 513 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181095+0000", 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4059999999999999e-06 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181106+0000", 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0868e-05 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181145+0000", 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9168000000000002e-05 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181157+0000", 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2343e-05 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181229+0000", 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1264000000000001e-05 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181415+0000", 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000186804 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181722+0000", 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030624200000000003 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.185844+0000", 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041219359999999997 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186379+0000", 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053546599999999996 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186495+0000", 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115575 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186583+0000", 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8128999999999996e-05 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:542 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.942530165000001, 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060375849999999998, 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 542 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234033+0000", 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8599999999999996e-07 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234038+0000", 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4830000000000001e-06 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234047+0000", 2024-09-16T12:59:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3109999999999995e-06 2024-09-16T12:59:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234053+0000", 2024-09-16T12:59:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6339999999999998e-06 2024-09-16T12:59:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.235355+0000", 2024-09-16T12:59:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013027589999999999 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.235617+0000", 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026182000000000001 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.236547+0000", 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093014599999999997 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239456+0000", 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029083289999999999 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239788+0000", 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033232900000000002 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239946+0000", 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015829700000000001 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.240070+0000", 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012349099999999999 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:571 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.867498274999999, 2024-09-16T12:59:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005551762, 2024-09-16T12:59:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 571 2024-09-16T12:59:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309066+0000", 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2060000000000001e-06 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309080+0000", 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3413e-05 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309114+0000", 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4180000000000001e-05 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309126+0000", 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2129e-05 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309312+0000", 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018575800000000001 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309477+0000", 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000164885 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.310054+0000", 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057696199999999998 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.311809+0000", 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001755047 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314482+0000", 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026735330000000001 2024-09-16T12:59:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314559+0000", 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6884000000000005e-05 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314616+0000", 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6765000000000003e-05 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:638 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.673326712000001, 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052595139999999999, 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 638 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503237+0000", 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.02e-06 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503248+0000", 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1114000000000001e-05 2024-09-16T12:59:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503263+0000", 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092999999999999e-05 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503273+0000", 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0413e-05 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.504835+0000", 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001562164 2024-09-16T12:59:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.505031+0000", 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000195465 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.505731+0000", 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070030899999999998 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507931+0000", 2024-09-16T12:59:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022001429999999999 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508299+0000", 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036765399999999997 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508405+0000", 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106368 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508495+0000", 2024-09-16T12:59:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9771000000000005e-05 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:746 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.407426966999999, 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052549399999999996, 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 746 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769137+0000", 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330000000000001e-06 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769144+0000", 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2660000000000004e-06 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769162+0000", 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7997000000000001e-05 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769178+0000", 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6305999999999999e-05 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.770704+0000", 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015260880000000001 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.770965+0000", 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026027800000000002 2024-09-16T12:59:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.771962+0000", 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099762399999999991 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.773232+0000", 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012702499999999999 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774136+0000", 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000903355 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774275+0000", 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139431 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774390+0000", 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011511200000000001 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:821 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.258514730999998, 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052264670000000003, 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 821 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918049+0000", 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.734e-06 2024-09-16T12:59:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918058+0000", 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918078+0000", 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0163e-05 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918100+0000", 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1396999999999999e-05 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.919700+0000", 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001600838 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.919878+0000", 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177276 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.920899+0000", 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010218219999999999 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.922776+0000", 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001876436 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923007+0000", 2024-09-16T12:59:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023069799999999999 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923160+0000", 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015317299999999999 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923274+0000", 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114298 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1077 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.743313038, 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062511429999999998, 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1077 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433250+0000", 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1370000000000001e-06 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433256+0000", 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433266+0000", 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0529e-05 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433278+0000", 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2363e-05 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.435866+0000", 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025878519999999999 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.436125+0000", 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002587 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.436955+0000", 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082961500000000002 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.438686+0000", 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017308930000000001 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439352+0000", 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066689400000000002 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439455+0000", 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102876 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439500+0000", 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5151000000000002e-05 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:3528 4.8 4:15914a01:::rbd_data.11b5c378985b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.158655109000001, 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051798410000000001, 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3528 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017909+0000", 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.283e-06 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017915+0000", 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4729999999999997e-06 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017941+0000", 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5910000000000001e-05 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017951+0000", 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0557000000000001e-05 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.018552+0000", 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060063100000000002 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.018663+0000", 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011089199999999999 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.023087+0000", 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044240950000000003 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:4787 4.15 4:a80ea28d:::rbd_header.11b5b1b9bb41:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.30636679, 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060070260000000004, 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4787 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870199+0000", 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2660000000000002e-06 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870212+0000", 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2794e-05 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870248+0000", 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6798999999999998e-05 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870266+0000", 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7288000000000001e-05 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870385+0000", 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119671 2024-09-16T12:59:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870394+0000", 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4300000000000006e-06 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.873802+0000", 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034079269999999998 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.873905+0000", 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102895 2024-09-16T12:59:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.874314+0000", 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040942099999999998 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.875454+0000", 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001139784 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876028+0000", 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057454900000000004 2024-09-16T12:59:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876132+0000", 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010318399999999999 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876203+0000", 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1018000000000003e-05 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5060 4.7 4:e6558db2:::rbd_data.11b517fb51dd.0000000000000002:9 [sparse-read 2097152~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.900962022, 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074022819999999996, 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5060 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275602+0000", 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1509999999999998e-06 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275611+0000", 2024-09-16T12:59:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0490000000000004e-06 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275632+0000", 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1115999999999999e-05 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275643+0000", 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1394e-05 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.279273+0000", 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003629546 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.279360+0000", 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7213000000000007e-05 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.283003+0000", 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003642813 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5061 4.7 4:e6558db2:::rbd_data.11b517fb51dd.0000000000000002:9 [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.900908994, 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073381729999999999, 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5061 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275654+0000", 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.074e-06 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275658+0000", 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6880000000000001e-06 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275668+0000", 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7839999999999998e-06 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275674+0000", 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9889999999999997e-06 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282791+0000", 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071176089999999996 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282884+0000", 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2085999999999999e-05 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282992+0000", 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010794300000000001 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8419 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.402892+0000", 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.773670785, 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040293592000000003, 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8419 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402894+0000", 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402907+0000", 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402940+0000", 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3436000000000003e-05 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402953+0000", 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2840000000000001e-05 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403104+0000", 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150712 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403202+0000", 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8403e-05 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.411015+0000", 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078133079999999997 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443018+0000", 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032002348999999999 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443075+0000", 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7703e-05 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443141+0000", 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5705999999999997e-05 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443185+0000", 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4152e-05 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:8419 4.7 e54/40 ondisk, result = 0)", 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.431024+0000", 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.7455382319999999, 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012172095000000001, 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431024+0000", 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431024+0000", 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431025+0000", 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3820000000000001e-06 2024-09-16T12:59:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431032+0000", 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4389999999999997e-06 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431054+0000", 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2008e-05 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431064+0000", 2024-09-16T12:59:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-05 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443061+0000", 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011997017 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443069+0000", 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7309999999999993e-06 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443196+0000", 2024-09-16T12:59:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012701800000000001 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8436 4.8 4:1430915d:::rbd_data.11b55eb65e78.0000000000000002:head [stat,truncate 14498372] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.493573+0000", 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.682989482, 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074513289999999996, 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8436 2024-09-16T12:59:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493573+0000", 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493573+0000", 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493575+0000", 2024-09-16T12:59:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4719999999999998e-06 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493587+0000", 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1792e-05 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493627+0000", 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9743999999999998e-05 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493644+0000", 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6796999999999999e-05 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493778+0000", 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133934 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.494050+0000", 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027250100000000003 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.494428+0000", 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037836199999999998 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.497239+0000", 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028101760000000002 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.500820+0000", 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003581608 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.500937+0000", 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011657799999999999 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.501024+0000", 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7365000000000001e-05 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8612 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.847921+0000", 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.328641121, 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039491146999999997, 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8612 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847921+0000", 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847921+0000", 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847924+0000", 2024-09-16T12:59:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0199999999999999e-06 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847936+0000", 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1579999999999999e-05 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847968+0000", 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2033000000000003e-05 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847986+0000", 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7839e-05 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.848125+0000", 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013962800000000001 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.848277+0000", 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015126199999999999 2024-09-16T12:59:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.856740+0000", 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084638479999999995 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887229+0000", 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030488491 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887253+0000", 2024-09-16T12:59:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4409e-05 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887347+0000", 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3467999999999993e-05 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887412+0000", 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5568999999999995e-05 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:8612 4.7 e54/40 ondisk, result = 0)", 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.880870+0000", 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2956926580000001, 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063651840000000003, 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880870+0000", 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880870+0000", 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880871+0000", 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.463e-06 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880877+0000", 2024-09-16T12:59:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0859999999999998e-06 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880898+0000", 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0979999999999999e-05 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880907+0000", 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2920000000000001e-06 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887172+0000", 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062649079999999996 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887211+0000", 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8695999999999997e-05 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887235+0000", 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3759000000000001e-05 2024-09-16T12:59:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8851 4.8 4:1430915d:::rbd_data.11b55eb65e78.0000000000000002:head [delete] snapc e=[e] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:38.091084+0000", 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.085478286000000001, 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057651029999999997, 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8851 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091084+0000", 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091084+0000", 2024-09-16T12:59:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091087+0000", 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.847e-06 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091097+0000", 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0502999999999999e-05 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091129+0000", 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1513000000000001e-05 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091143+0000", 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4535999999999999e-05 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091230+0000", 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6670999999999997e-05 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091356+0000", 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012561000000000001 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091599+0000", 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000243685 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.093310+0000", 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017106689999999999 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.096773+0000", 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034631760000000001 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.096817+0000", 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3513000000000002e-05 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.096849+0000", 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2379999999999998e-05 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:38.293 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:38.294 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:38.294 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:38.294 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:38.294 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:38.294 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:38.295 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:38.283+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:38.295 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:38.298 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:38.319 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:38.321 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:38.321 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:38.322 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:38.322 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:38.322 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:38.322 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:38.322 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:38.322 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:38.351 INFO:teuthology.orchestra.run.smithi042.stdout:1613 write 0x5c952e8 thru 0x5c988fb (0x3614 bytes) 2024-09-16T12:59:38.387 INFO:teuthology.orchestra.run.smithi042.stdout:1614 trunc from 0xdacc43e to 0x2af55b5 2024-09-16T12:59:38.395 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:38.387+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:38.410 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:38.410 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:38.420 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:38.452 INFO:teuthology.orchestra.run.smithi042.stdout:1615 write 0xe91c17b thru 0xe922832 (0x66b8 bytes) 2024-09-16T12:59:38.461 INFO:teuthology.orchestra.run.smithi042.stdout:1616 write 0x60cbf01 thru 0x60d99b4 (0xdab4 bytes) 2024-09-16T12:59:38.463 INFO:teuthology.orchestra.run.smithi042.stdout:1617 read 0x12c4af9 thru 0x12d3553 (0xea5b bytes) 2024-09-16T12:59:38.465 INFO:teuthology.orchestra.run.smithi042.stdout:1618 punch from 0x49aa028 to 0x49b3f60, (0x9f38 bytes) 2024-09-16T12:59:38.465 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:38.465 INFO:teuthology.orchestra.run.smithi042.stdout:1620 punch from 0x31e8b1e to 0x31eb0ef, (0x25d1 bytes) 2024-09-16T12:59:38.465 INFO:teuthology.orchestra.run.smithi042.stdout:1621 writesame 0xc53d310 thru 0xc547dc3 data_size 0x4c(0xaab4 bytes) 2024-09-16T12:59:38.485 INFO:teuthology.orchestra.run.smithi042.stdout:1623 trunc from 0xe922833 to 0x8cff779 2024-09-16T12:59:38.495 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:38.487+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:38.510 INFO:teuthology.orchestra.run.smithi042.stdout:1624 punch from 0x42df668 to 0x42ed5af, (0xdf47 bytes) 2024-09-16T12:59:38.521 INFO:teuthology.orchestra.run.smithi042.stdout:1625 trunc from 0x8cff779 to 0x2a2fa41 2024-09-16T12:59:38.523 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:38.523 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:38.534 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:38.546 INFO:teuthology.orchestra.run.smithi042.stdout:1626 trunc from 0x2a2fa41 to 0x585ecd1 2024-09-16T12:59:38.556 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:38.556 INFO:teuthology.orchestra.run.smithi042.stdout:1628 read 0x4753d06 thru 0x475cb90 (0x8e8b bytes) 2024-09-16T12:59:38.557 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:38.557 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:38.557 INFO:teuthology.orchestra.run.smithi042.stdout:1631 writesame 0x2d4cffe thru 0x2d51a2a data_size 0x1(0x4a2d bytes) 2024-09-16T12:59:38.573 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:38.573 INFO:teuthology.orchestra.run.smithi042.stdout:1633 read 0xa57d2f thru 0xa5ab3b (0x2e0d bytes) 2024-09-16T12:59:38.573 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:38.573 INFO:teuthology.orchestra.run.smithi042.stdout:1635 write 0x2564892 thru 0x256af8c (0x66fb bytes) 2024-09-16T12:59:38.596 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:38.591+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:38.603 INFO:teuthology.orchestra.run.smithi042.stdout:1636 writesame 0xc38f97f thru 0xc39da54 data_size 0x35(0xe0d6 bytes) 2024-09-16T12:59:38.603 INFO:teuthology.orchestra.run.smithi042.stdout:1637 writesame 0xae89888 thru 0xae8a12c data_size 0x1(0x8a5 bytes) 2024-09-16T12:59:38.610 INFO:teuthology.orchestra.run.smithi042.stdout:1638 punch from 0x6037de0 to 0x603f8a9, (0x7ac9 bytes) 2024-09-16T12:59:38.610 INFO:teuthology.orchestra.run.smithi042.stdout:1639 clone 12 order 18 su 16384 sc 13 2024-09-16T12:59:38.635 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:38.635 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:38.644 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:38.697 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:38.687+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:38.744 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:38.744 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:38.753 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:38.798 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:38.795+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:38.854 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:38.855 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:38.864 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:38.898 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:38.895+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:38.946 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T12:59:38.947 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T12:59:38.947 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-16T12:59:38.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-16T12:59:38.963 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:38.963 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:38.973 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:38.999 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:38.995+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:39.096 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone11 from 0xc39da55 (overlap 0x2a2fa41) to 0x1d87bfa 2024-09-16T12:59:39.098 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:39.099 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:39.099 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:39.091+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:39.108 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:39.200 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:39.191+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:39.202 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:39.202 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:39.213 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:39.233 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-16T12:59:39.236 INFO:teuthology.orchestra.run.smithi042.stdout:1640 read 0x34b08d6 thru 0x34bdea1 (0xd5cc bytes) 2024-09-16T12:59:39.238 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:39.239 INFO:teuthology.orchestra.run.smithi042.stdout:1642 write 0x3819077 thru 0x382471b (0xb6a5 bytes) 2024-09-16T12:59:39.261 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:39.276 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-16T12:59:39.267+0000 7f5982d84700 -1 mon.a@0(leader).osd e56 definitely_dead 0 2024-09-16T12:59:39.277 INFO:teuthology.orchestra.run.smithi042.stdout:1644 trunc from 0xc39da55 to 0x29db52d 2024-09-16T12:59:39.301 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:39.291+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:39.323 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:39.323 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:39.334 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:39.334 INFO:teuthology.orchestra.run.smithi042.stdout:1645 write 0x5c31cd8 thru 0x5c3bf05 (0xa22e bytes) 2024-09-16T12:59:39.348 INFO:teuthology.orchestra.run.smithi042.stdout:1646 trunc from 0x5c3bf06 to 0x413586d 2024-09-16T12:59:39.395 INFO:teuthology.orchestra.run.smithi042.stdout:1650 write 0xd9bfcf1 thru 0xd9c9dc5 (0xa0d5 bytes) 2024-09-16T12:59:39.401 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:39.395+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:39.409 INFO:teuthology.orchestra.run.smithi042.stdout:1651 write 0x28fd1ca thru 0x2907442 (0xa279 bytes) 2024-09-16T12:59:39.412 INFO:teuthology.orchestra.run.smithi042.stdout:1652 read 0x19dbe5d thru 0x19e8da5 (0xcf49 bytes) 2024-09-16T12:59:39.413 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:39.413 INFO:teuthology.orchestra.run.smithi042.stdout:1654 read 0xb57245c thru 0xb5777fd (0x53a2 bytes) 2024-09-16T12:59:39.413 INFO:teuthology.orchestra.run.smithi042.stdout:1656 read 0x366c85d thru 0x3673a79 (0x721d bytes) 2024-09-16T12:59:39.413 INFO:teuthology.orchestra.run.smithi042.stdout:1657 writesame 0x1020573 thru 0x1020f10 data_size 0x1(0x99e bytes) 2024-09-16T12:59:39.421 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:39.421 INFO:teuthology.orchestra.run.smithi042.stdout:1660 read 0xb2cd832 thru 0xb2d4019 (0x67e8 bytes) 2024-09-16T12:59:39.421 INFO:teuthology.orchestra.run.smithi042.stdout:1663 read 0xd570d78 thru 0xd57cc45 (0xbece bytes) 2024-09-16T12:59:39.421 INFO:teuthology.orchestra.run.smithi042.stdout:1665 read 0x78791a5 thru 0x788175c (0x85b8 bytes) 2024-09-16T12:59:39.421 INFO:teuthology.orchestra.run.smithi042.stdout:1666 write 0x8176a20 thru 0x8182bf1 (0xc1d2 bytes) 2024-09-16T12:59:39.432 INFO:teuthology.orchestra.run.smithi042.stdout:1667 write 0x3066226 thru 0x30722e7 (0xc0c2 bytes) 2024-09-16T12:59:39.437 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:39.437 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:39.442 INFO:teuthology.orchestra.run.smithi042.stdout:1668 write 0x8e03f42 thru 0x8e0401a (0xd9 bytes) 2024-09-16T12:59:39.448 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:39.449 INFO:teuthology.orchestra.run.smithi042.stdout:1669 punch from 0x3bef06 to 0x3c6e9e, (0x7f98 bytes) 2024-09-16T12:59:39.449 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:39.449 INFO:teuthology.orchestra.run.smithi042.stdout:1672 write 0xd5a2619 thru 0xd5a9e98 (0x7880 bytes) 2024-09-16T12:59:39.456 INFO:teuthology.orchestra.run.smithi042.stdout:1673 write 0x145a02b thru 0x145dc15 (0x3beb bytes) 2024-09-16T12:59:39.465 INFO:teuthology.orchestra.run.smithi042.stdout:1674 write 0x4d63309 thru 0x4d69499 (0x6191 bytes) 2024-09-16T12:59:39.472 INFO:teuthology.orchestra.run.smithi042.stdout:1675 punch from 0x14fee2c to 0x1502c78, (0x3e4c bytes) 2024-09-16T12:59:39.480 INFO:teuthology.orchestra.run.smithi042.stdout:1676 trunc from 0xd9c9dc6 to 0x8bc0219 2024-09-16T12:59:39.502 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:39.491+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:39.512 INFO:teuthology.orchestra.run.smithi042.stdout:1677 write 0x126ca09 thru 0x1276775 (0x9d6d bytes) 2024-09-16T12:59:39.525 INFO:teuthology.orchestra.run.smithi042.stdout:1679 read 0x83cb24f thru 0x83d0ed2 (0x5c84 bytes) 2024-09-16T12:59:39.529 INFO:teuthology.orchestra.run.smithi042.stdout:1681 trunc from 0x8bc0219 to 0x65dffba 2024-09-16T12:59:39.555 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:39.555 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:39.566 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:39.568 INFO:teuthology.orchestra.run.smithi042.stdout:1682 write 0xe1125d1 thru 0xe11a057 (0x7a87 bytes) 2024-09-16T12:59:39.580 INFO:teuthology.orchestra.run.smithi042.stdout:1683 punch from 0x7e82c8d to 0x7e8f98c, (0xccff bytes) 2024-09-16T12:59:39.580 INFO:teuthology.orchestra.run.smithi042.stdout:1685 read 0x50eb59b thru 0x50f364f (0x80b5 bytes) 2024-09-16T12:59:39.581 INFO:teuthology.orchestra.run.smithi042.stdout:1687 punch from 0x4018d85 to 0x401d4e5, (0x4760 bytes) 2024-09-16T12:59:39.581 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:39.581 INFO:teuthology.orchestra.run.smithi042.stdout:1690 punch from 0x5b3a484 to 0x5b4051e, (0x609a bytes) 2024-09-16T12:59:39.581 INFO:teuthology.orchestra.run.smithi042.stdout:1693 read 0x8afa862 thru 0x8b0a1de (0xf97d bytes) 2024-09-16T12:59:39.581 INFO:teuthology.orchestra.run.smithi042.stdout:1695 punch from 0xeb4ee0 to 0xeba821, (0x5941 bytes) 2024-09-16T12:59:39.581 INFO:teuthology.orchestra.run.smithi042.stdout:1696 write 0xdab6ac7 thru 0xdab6f31 (0x46b bytes) 2024-09-16T12:59:39.586 INFO:teuthology.orchestra.run.smithi042.stdout:1697 read 0x775f12 thru 0x7813a3 (0xb492 bytes) 2024-09-16T12:59:39.588 INFO:teuthology.orchestra.run.smithi042.stdout:1699 writesame 0x97d7e40 thru 0x97e29d1 data_size 0x1(0xab92 bytes) 2024-09-16T12:59:39.602 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:39.599+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:39.620 INFO:teuthology.orchestra.run.smithi042.stdout:1700 write 0xe4dcabf thru 0xe4eb0c8 (0xe60a bytes) 2024-09-16T12:59:39.633 INFO:teuthology.orchestra.run.smithi042.stdout:1702 writesame 0x803c794 thru 0x80416a9 data_size 0x2(0x4f16 bytes) 2024-09-16T12:59:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:1703 read 0x81270a6 thru 0x813151e (0xa479 bytes) 2024-09-16T12:59:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:1704 read 0x83e190b thru 0x83ee73d (0xce33 bytes) 2024-09-16T12:59:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:39.647 INFO:teuthology.orchestra.run.smithi042.stdout:1707 writesame 0x45e0cd4 thru 0x45e3cd8 data_size 0x1(0x3005 bytes) 2024-09-16T12:59:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:39.682 INFO:teuthology.orchestra.run.smithi042.stdout:1708 trunc from 0xe4eb0c9 to 0x362e23f 2024-09-16T12:59:39.686 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:39.730 INFO:teuthology.orchestra.run.smithi042.stdout:1709 write 0xb500a86 thru 0xb505630 (0x4bab bytes) 2024-09-16T12:59:39.730 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:39.723+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:39.738 INFO:teuthology.orchestra.run.smithi042.stdout:1711 read 0x5861201 thru 0x586d97f (0xc77f bytes) 2024-09-16T12:59:39.738 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:39.738 INFO:teuthology.orchestra.run.smithi042.stdout:1713 writesame 0x43f39d6 thru 0x43f679e data_size 0x1(0x2dc9 bytes) 2024-09-16T12:59:39.755 INFO:teuthology.orchestra.run.smithi042.stdout:1714 punch from 0x9c7db82 to 0x9c7f1b5, (0x1633 bytes) 2024-09-16T12:59:39.755 INFO:teuthology.orchestra.run.smithi042.stdout:1715 read 0x5c5c666 thru 0x5c5d1bf (0xb5a bytes) 2024-09-16T12:59:39.755 INFO:teuthology.orchestra.run.smithi042.stdout:1717 clone 13 order 23 su 4194304 sc 12 2024-09-16T12:59:39.805 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:39.795+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:39.833 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:39.833 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:39.842 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:39.905 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:39.895+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:39.985 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T12:59:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:marked in osd.1. 2024-09-16T12:59:40.006 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:39.999+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:40.006 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:40.006 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:40.009 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-16T12:59:40.015 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:40.108 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:40.103+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:40.160 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:40.160 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:40.169 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:40.208 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:40.199+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:40.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:40.318 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:40.311+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:40.336 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:40.410 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:40.407+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:40.421 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:40.421 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:40.432 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:40.471 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:40.492 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:40.511 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:40.503+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:40.515 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:40.526 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:40.526 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:40.536 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:40.536 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:40.558 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:40.579 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:40.601 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:40.612 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:40.603+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:40.625 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:40.649 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:40.649 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:40.649 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:40.649 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:40.659 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:40.713 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:40.707+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:40.754 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:40.754 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:40.754 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:40.755 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:40.755 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:40.763 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:40.813 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:40.803+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:40.870 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:590 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.526420759000001, 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034258392999999998, 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 590 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339271+0000", 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339281+0000", 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6670000000000008e-06 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339307+0000", 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5907000000000001e-05 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339317+0000", 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0356999999999999e-05 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339412+0000", 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4248999999999998e-05 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339533+0000", 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012184900000000001 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.347489+0000", 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079559599999999998 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373423+0000", 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025933971 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373435+0000", 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1924e-05 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373492+0000", 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6842999999999998e-05 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373528+0000", 2024-09-16T12:59:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1298 4.18 4:1f49b46c:::rbd_object_map.11b53140b009:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.076549+0000", 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.789141519000001, 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012581233000000001, 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.877 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1298 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076549+0000", 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076549+0000", 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076550+0000", 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9599999999999998e-07 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076554+0000", 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8289999999999998e-06 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076565+0000", 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0446e-05 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.076569+0000", 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5220000000000001e-06 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.082476+0000", 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059068799999999998 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.082603+0000", 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012666399999999999 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.083102+0000", 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049894900000000005 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.087350+0000", 2024-09-16T12:59:40.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042485379999999996 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.088968+0000", 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001617804 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089056+0000", 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7690999999999998e-05 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.089130+0000", 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4014e-05 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5063 4.13 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22711298~51120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.556275961000001, 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111447293, 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.883 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5063 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309416+0000", 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358861+0000", 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049444717999999999 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358893+0000", 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2279e-05 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358902+0000", 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4390000000000006e-06 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359062+0000", 2024-09-16T12:59:40.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016014100000000001 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359330+0000", 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026734699999999997 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.362566+0000", 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032358769999999999 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.417541+0000", 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054975345000000002 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420706+0000", 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031651380000000001 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420790+0000", 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4283999999999995e-05 2024-09-16T12:59:40.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420862+0000", 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1138999999999998e-05 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5225 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.804570+0000", 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.061120641, 2024-09-16T12:59:40.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027003329999999999, 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5225 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804571+0000", 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6309999999999999e-06 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804579+0000", 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804600+0000", 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1114000000000001e-05 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804613+0000", 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3040000000000001e-05 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804713+0000", 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9854999999999999e-05 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804844+0000", 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130963 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.810955+0000", 2024-09-16T12:59:40.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061105880000000001 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831495+0000", 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020540467 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831507+0000", 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831544+0000", 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7413000000000002e-05 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831573+0000", 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8836e-05 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5504 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:40.895 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.468273+0000", 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.397417035, 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020881448, 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5504 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468276+0000", 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8370000000000001e-06 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468290+0000", 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.383e-05 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468327+0000", 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468341+0000", 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.331e-05 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468439+0000", 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8746999999999995e-05 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468593+0000", 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015343999999999999 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.472806+0000", 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042129849999999998 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488823+0000", 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016017686999999999 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488936+0000", 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112074 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489029+0000", 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3801000000000005e-05 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489155+0000", 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125248 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8352 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.280585+0000", 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5851052450000003, 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014415468000000001, 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8352 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280585+0000", 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280585+0000", 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280587+0000", 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0229999999999999e-06 2024-09-16T12:59:40.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280597+0000", 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6139999999999998e-06 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280617+0000", 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9969000000000001e-05 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280626+0000", 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.290184+0000", 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009558024 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.290251+0000", 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6210000000000005e-05 2024-09-16T12:59:40.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.290656+0000", 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040517800000000002 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294871+0000", 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042150529999999999 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294885+0000", 2024-09-16T12:59:40.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4154999999999999e-05 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294956+0000", 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1101999999999998e-05 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295000+0000", 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4339999999999999e-05 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.907 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8353 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.280701+0000", 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5849890740000001, 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015810297000000001, 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8353 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.908 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280701+0000", 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280701+0000", 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280702+0000", 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5000000000000001e-07 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280708+0000", 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6130000000000003e-06 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280730+0000", 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.211e-05 2024-09-16T12:59:40.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280737+0000", 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.323e-06 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.291663+0000", 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010926282000000001 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.291729+0000", 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5908000000000007e-05 2024-09-16T12:59:40.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.291994+0000", 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026483799999999999 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296145+0000", 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004150969 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296448+0000", 2024-09-16T12:59:40.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030309 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296484+0000", 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5488999999999999e-05 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296511+0000", 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7725e-05 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8354 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.281367+0000", 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5843229650000001, 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016884742000000001, 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8354 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281370+0000", 2024-09-16T12:59:40.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3159999999999999e-06 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281382+0000", 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281413+0000", 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1519000000000001e-05 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281435+0000", 2024-09-16T12:59:40.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1812999999999999e-05 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292521+0000", 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011085915999999999 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292628+0000", 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107311 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293108+0000", 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047976600000000001 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297972+0000", 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048640200000000001 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298174+0000", 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020222 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298226+0000", 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2058000000000002e-05 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298252+0000", 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5743000000000001e-05 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.919 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8355 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.282597+0000", 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.583093066, 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015824709999999999, 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8355 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282597+0000", 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282597+0000", 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282600+0000", 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3870000000000002e-06 2024-09-16T12:59:40.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282613+0000", 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3733e-05 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282634+0000", 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0766000000000001e-05 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282645+0000", 2024-09-16T12:59:40.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0783e-05 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293789+0000", 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011144664 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293857+0000", 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7267999999999994e-05 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294144+0000", 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000287025 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297979+0000", 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003834811 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298314+0000", 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000335092 2024-09-16T12:59:40.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298371+0000", 2024-09-16T12:59:40.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7046999999999998e-05 2024-09-16T12:59:40.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298422+0000", 2024-09-16T12:59:40.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1134000000000002e-05 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8357 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.285472+0000", 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5802182499999997, 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015109900000000001, 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8357 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285472+0000", 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285472+0000", 2024-09-16T12:59:40.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285474+0000", 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9039999999999999e-06 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285481+0000", 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3640000000000001e-06 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285495+0000", 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3963e-05 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285503+0000", 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.712e-06 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295041+0000", 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095385009999999996 2024-09-16T12:59:40.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295111+0000", 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0097000000000003e-05 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295515+0000", 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040313399999999998 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297985+0000", 2024-09-16T12:59:40.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024699729999999999 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.300526+0000", 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025414029999999998 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.300560+0000", 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4e-05 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.300582+0000", 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1849e-05 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8580 4.e 4:746c7c9d:::rbd_data.11b55eb65e78.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.744435+0000", 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.1212549190000001, 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012615856, 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.932 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8580 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744435+0000", 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744435+0000", 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744437+0000", 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1830000000000001e-06 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744449+0000", 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1800000000000001e-05 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744485+0000", 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5918999999999999e-05 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744503+0000", 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7759999999999999e-05 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744612+0000", 2024-09-16T12:59:40.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109368 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.744744+0000", 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131509 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.747374+0000", 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002629981 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.756812+0000", 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094380720000000005 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.756836+0000", 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3955e-05 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.756930+0000", 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4561999999999993e-05 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.757051+0000", 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120747 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8674 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009917+0000", 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8557729649999999, 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01501186, 2024-09-16T12:59:40.938 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8674 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009917+0000", 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009917+0000", 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009918+0000", 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009920+0000", 2024-09-16T12:59:40.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.379e-06 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009924+0000", 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5249999999999999e-06 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009928+0000", 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1200000000000002e-06 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017969+0000", 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080410150000000003 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.018191+0000", 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022191999999999999 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.019035+0000", 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084407199999999996 2024-09-16T12:59:40.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.022090+0000", 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030553849999999999 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024810+0000", 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027204009999999999 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024878+0000", 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7704000000000001e-05 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024929+0000", 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0983000000000003e-05 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.944 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8675 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009930+0000", 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.855759795, 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015359810999999999, 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8675 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009930+0000", 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009930+0000", 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009931+0000", 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6100000000000002e-07 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009934+0000", 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8559999999999998e-06 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009938+0000", 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5369999999999999e-06 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009942+0000", 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8970000000000001e-06 2024-09-16T12:59:40.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020310+0000", 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010367833 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020475+0000", 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016531400000000001 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.021011+0000", 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053554600000000003 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025005+0000", 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039943360000000002 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025130+0000", 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125165 2024-09-16T12:59:40.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025226+0000", 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5753000000000003e-05 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025290+0000", 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4213000000000003e-05 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8676 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8557452899999998, 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015451705, 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8676 2024-09-16T12:59:40.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009948+0000", 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6249999999999999e-06 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009952+0000", 2024-09-16T12:59:40.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3449999999999999e-06 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009955+0000", 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0510000000000001e-06 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.022130+0000", 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012175002000000001 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.022734+0000", 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060358299999999998 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023097+0000", 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036315300000000001 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025115+0000", 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020180720000000001 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025305+0000", 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018960299999999999 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025342+0000", 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7684e-05 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025397+0000", 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4231000000000003e-05 2024-09-16T12:59:40.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8677 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009958+0000", 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8557319950000002, 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01642155, 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8677 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T12:59:40.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009959+0000", 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2899999999999999e-07 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009961+0000", 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3760000000000002e-06 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009965+0000", 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009968+0000", 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1190000000000001e-06 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023570+0000", 2024-09-16T12:59:40.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013601317999999999 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023695+0000", 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012563400000000001 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024191+0000", 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000495434 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026285+0000", 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020948360000000001 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026295+0000", 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8889999999999997e-06 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026338+0000", 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2253999999999999e-05 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026380+0000", 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2153000000000001e-05 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8678 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009972+0000", 2024-09-16T12:59:40.963 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.855717743, 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020138508999999999, 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8678 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T12:59:40.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009973+0000", 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3000000000000002e-07 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009975+0000", 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4739999999999999e-06 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009980+0000", 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4399999999999998e-06 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009983+0000", 2024-09-16T12:59:40.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1209999999999998e-06 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025464+0000", 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015480918 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025536+0000", 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2212000000000006e-05 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025768+0000", 2024-09-16T12:59:40.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000232186 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029964+0000", 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041957899999999996 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029981+0000", 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6634e-05 2024-09-16T12:59:40.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030070+0000", 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9487999999999994e-05 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030111+0000", 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0915999999999999e-05 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8681 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.017890+0000", 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.847800559, 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013352773, 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8681 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017890+0000", 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017890+0000", 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017892+0000", 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1830000000000001e-06 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017905+0000", 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2782000000000001e-05 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017928+0000", 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3828e-05 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017940+0000", 2024-09-16T12:59:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1756999999999999e-05 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026416+0000", 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084759359999999999 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026489+0000", 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2618000000000002e-05 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026837+0000", 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000348389 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030134+0000", 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032973630000000002 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031172+0000", 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001037846 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031212+0000", 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9242999999999997e-05 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031242+0000", 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0827999999999997e-05 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8682 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.020416+0000", 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8452739249999999, 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013951481, 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8682 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020416+0000", 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020416+0000", 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020418+0000", 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6270000000000001e-06 2024-09-16T12:59:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020427+0000", 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5380000000000008e-06 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020444+0000", 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6770999999999999e-05 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020455+0000", 2024-09-16T12:59:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1088e-05 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.027337+0000", 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068815229999999996 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.027426+0000", 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8727000000000006e-05 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.027954+0000", 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052817300000000005 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.033199+0000", 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052455879999999998 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.034274+0000", 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010743759999999999 2024-09-16T12:59:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.034311+0000", 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7314999999999998e-05 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.034368+0000", 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6755000000000001e-05 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8686 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.023953+0000", 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8417373939999999, 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012887233, 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8686 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023953+0000", 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023953+0000", 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023954+0000", 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1179999999999999e-06 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023960+0000", 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3740000000000003e-06 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023971+0000", 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0274e-05 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023977+0000", 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.353e-06 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030172+0000", 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061950809999999999 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030237+0000", 2024-09-16T12:59:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4844999999999993e-05 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030529+0000", 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029214699999999998 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.035705+0000", 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051761699999999999 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.036741+0000", 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010355480000000001 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.036796+0000", 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5460000000000001e-05 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.036840+0000", 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3863000000000003e-05 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8718 4.1b 4:dbb4ad27:::rbd_data.11b55eb65e78.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.086287+0000", 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.779402782, 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028260378999999999, 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8718 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086290+0000", 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1950000000000002e-06 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086300+0000", 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0458999999999999e-05 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086331+0000", 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1117e-05 2024-09-16T12:59:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086344+0000", 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3191e-05 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086423+0000", 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8288999999999995e-05 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086530+0000", 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010720799999999999 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.091612+0000", 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050817429999999997 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114274+0000", 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022662832000000001 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114386+0000", 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011155899999999999 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114475+0000", 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9043999999999997e-05 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114548+0000", 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2742000000000003e-05 2024-09-16T12:59:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:40.994 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:40.995 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:40.996 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:40.991+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:40.996 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T12:59:41.091 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:41.091 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:41.091 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:41.091 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:41.096 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:41.087+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:41.101 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:41.197 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:41.197 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:41.197 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:41.197 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:41.197 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:41.198 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:41.187+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:41.206 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:41.299 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:41.295+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:41.304 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.670592653, 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006297965, 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629310+0000", 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629315+0000", 2024-09-16T12:59:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7080000000000002e-06 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629324+0000", 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0250000000000008e-06 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629340+0000", 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.59e-05 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629351+0000", 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1236999999999999e-05 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629565+0000", 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000213482 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.629582+0000", 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7482999999999998e-05 2024-09-16T12:59:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.631980+0000", 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023977830000000001 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.632247+0000", 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000266867 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.632356+0000", 2024-09-16T12:59:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010894 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.633744+0000", 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013876240000000001 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635507+0000", 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001763288 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635562+0000", 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5096999999999998e-05 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:53.635608+0000", 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5531000000000002e-05 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T12:59:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.658527055999997, 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063927960000000001, 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641375+0000", 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641378+0000", 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6429999999999999e-06 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641390+0000", 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1912e-05 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641422+0000", 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2666999999999997e-05 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641448+0000", 2024-09-16T12:59:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5636e-05 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641623+0000", 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017479299999999999 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.641641+0000", 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8369999999999999e-05 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.644629+0000", 2024-09-16T12:59:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00298745 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.644995+0000", 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000366688 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.645348+0000", 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035283200000000003 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647159+0000", 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018104760000000001 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647631+0000", 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047253799999999999 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647708+0000", 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6715e-05 2024-09-16T12:59:41.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:54.647768+0000", 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0075999999999999e-05 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4272.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.650947097, 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006003988, 2024-09-16T12:59:41.318 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.319 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4272", 2024-09-16T12:59:41.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/2395197261", 2024-09-16T12:59:41.319 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 7 2024-09-16T12:59:41.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.320 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:41.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648955+0000", 2024-09-16T12:59:41.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648958+0000", 2024-09-16T12:59:41.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0280000000000001e-06 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.648970+0000", 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1374e-05 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649002+0000", 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2298999999999997e-05 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649014+0000", 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2442e-05 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649099+0000", 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4655999999999997e-05 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.649107+0000", 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0800000000000006e-06 2024-09-16T12:59:41.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.652487+0000", 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003380241 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.652733+0000", 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024601000000000001 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.653091+0000", 2024-09-16T12:59:41.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035755599999999998 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654603+0000", 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001512163 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654795+0000", 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000192179 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654875+0000", 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0211999999999997e-05 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:55.654959+0000", 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3747999999999998e-05 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:513 4.8 4:132f6b09:::rbd_data.11b59d02cc38.000000000000008c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.118809735999999, 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054902010000000001, 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 513 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181093+0000", 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181095+0000", 2024-09-16T12:59:41.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4059999999999999e-06 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181106+0000", 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0868e-05 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181145+0000", 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9168000000000002e-05 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181157+0000", 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2343e-05 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181229+0000", 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1264000000000001e-05 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181415+0000", 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000186804 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.181722+0000", 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030624200000000003 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.185844+0000", 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041219359999999997 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186379+0000", 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053546599999999996 2024-09-16T12:59:41.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186495+0000", 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115575 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.186583+0000", 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8128999999999996e-05 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:542 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.065870113999999, 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060375849999999998, 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 542 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234032+0000", 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234033+0000", 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8599999999999996e-07 2024-09-16T12:59:41.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234038+0000", 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4830000000000001e-06 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234047+0000", 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3109999999999995e-06 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.234053+0000", 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6339999999999998e-06 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.235355+0000", 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013027589999999999 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.235617+0000", 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026182000000000001 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.236547+0000", 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093014599999999997 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239456+0000", 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029083289999999999 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239788+0000", 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033232900000000002 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.239946+0000", 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015829700000000001 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.240070+0000", 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012349099999999999 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.338 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:571 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.990838224000001, 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005551762, 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 571 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.339 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309064+0000", 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309066+0000", 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2060000000000001e-06 2024-09-16T12:59:41.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309080+0000", 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3413e-05 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309114+0000", 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4180000000000001e-05 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309126+0000", 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2129e-05 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309312+0000", 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018575800000000001 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.309477+0000", 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000164885 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.310054+0000", 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057696199999999998 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.311809+0000", 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001755047 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314482+0000", 2024-09-16T12:59:41.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026735330000000001 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314559+0000", 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6884000000000005e-05 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.314616+0000", 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6765000000000003e-05 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.344 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:638 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.796666661, 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052595139999999999, 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 638 2024-09-16T12:59:41.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503236+0000", 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503237+0000", 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.02e-06 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503248+0000", 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1114000000000001e-05 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503263+0000", 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092999999999999e-05 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.503273+0000", 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0413e-05 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.504835+0000", 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001562164 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.505031+0000", 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000195465 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.505731+0000", 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070030899999999998 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.507931+0000", 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022001429999999999 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508299+0000", 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036765399999999997 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508405+0000", 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106368 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.508495+0000", 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9771000000000005e-05 2024-09-16T12:59:41.350 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:746 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.530766916000001, 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052549399999999996, 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.351 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 746 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769135+0000", 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769137+0000", 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330000000000001e-06 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769144+0000", 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2660000000000004e-06 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769162+0000", 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7997000000000001e-05 2024-09-16T12:59:41.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.769178+0000", 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6305999999999999e-05 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.770704+0000", 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015260880000000001 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.770965+0000", 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026027800000000002 2024-09-16T12:59:41.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.771962+0000", 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099762399999999991 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.773232+0000", 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012702499999999999 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774136+0000", 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000903355 2024-09-16T12:59:41.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774275+0000", 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139431 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.774390+0000", 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011511200000000001 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:821 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.38185468, 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052264670000000003, 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 821 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918048+0000", 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918049+0000", 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.734e-06 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918058+0000", 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918078+0000", 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0163e-05 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.918100+0000", 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1396999999999999e-05 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.919700+0000", 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001600838 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.919878+0000", 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177276 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.920899+0000", 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010218219999999999 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.922776+0000", 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001876436 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923007+0000", 2024-09-16T12:59:41.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023069799999999999 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923160+0000", 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015317299999999999 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.923274+0000", 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114298 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1077 4.8 4:15b38636:::rbd_object_map.11b59d02cc38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.866652986999998, 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062511429999999998, 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1077 2024-09-16T12:59:41.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433249+0000", 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433250+0000", 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1370000000000001e-06 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433256+0000", 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433266+0000", 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0529e-05 2024-09-16T12:59:41.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.433278+0000", 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2363e-05 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.435866+0000", 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025878519999999999 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.436125+0000", 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002587 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.436955+0000", 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082961500000000002 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.438686+0000", 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017308930000000001 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439352+0000", 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066689400000000002 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439455+0000", 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102876 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:18.439500+0000", 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5151000000000002e-05 2024-09-16T12:59:41.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:3528 4.8 4:15914a01:::rbd_data.11b5c378985b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.281995058, 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051798410000000001, 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3528 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017907+0000", 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017909+0000", 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.283e-06 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017915+0000", 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4729999999999997e-06 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017941+0000", 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5910000000000001e-05 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.017951+0000", 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0557000000000001e-05 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.018552+0000", 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060063100000000002 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.018663+0000", 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011089199999999999 2024-09-16T12:59:41.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.023087+0000", 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044240950000000003 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:4787 4.15 4:a80ea28d:::rbd_header.11b5b1b9bb41:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.429706739, 2024-09-16T12:59:41.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060070260000000004, 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4787 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870196+0000", 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870199+0000", 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2660000000000002e-06 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870212+0000", 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2794e-05 2024-09-16T12:59:41.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870248+0000", 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6798999999999998e-05 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870266+0000", 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7288000000000001e-05 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870385+0000", 2024-09-16T12:59:41.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119671 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.870394+0000", 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4300000000000006e-06 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.873802+0000", 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034079269999999998 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.873905+0000", 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102895 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.874314+0000", 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040942099999999998 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.875454+0000", 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001139784 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876028+0000", 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057454900000000004 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876132+0000", 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010318399999999999 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:26.876203+0000", 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1018000000000003e-05 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.379 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5060 4.7 4:e6558db2:::rbd_data.11b517fb51dd.0000000000000002:9 [sparse-read 2097152~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.024301971, 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074022819999999996, 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.380 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5060 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275600+0000", 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275602+0000", 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1509999999999998e-06 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275611+0000", 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0490000000000004e-06 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275632+0000", 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1115999999999999e-05 2024-09-16T12:59:41.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275643+0000", 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1394e-05 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.279273+0000", 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003629546 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.279360+0000", 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7213000000000007e-05 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.283003+0000", 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003642813 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5061 4.7 4:e6558db2:::rbd_data.11b517fb51dd.0000000000000002:9 [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:41.384 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.024248943, 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073381729999999999, 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5061 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:41.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275653+0000", 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275654+0000", 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.074e-06 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275658+0000", 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6880000000000001e-06 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275668+0000", 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7839999999999998e-06 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.275674+0000", 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9889999999999997e-06 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282791+0000", 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071176089999999996 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282884+0000", 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2085999999999999e-05 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.282992+0000", 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010794300000000001 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.388 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8419 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.402892+0000", 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8970107340000002, 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040293592000000003, 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8419 2024-09-16T12:59:41.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402894+0000", 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402907+0000", 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402940+0000", 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3436000000000003e-05 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402953+0000", 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2840000000000001e-05 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403104+0000", 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150712 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403202+0000", 2024-09-16T12:59:41.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8403e-05 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.411015+0000", 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078133079999999997 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443018+0000", 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032002348999999999 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443075+0000", 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7703e-05 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443141+0000", 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5705999999999997e-05 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443185+0000", 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4152e-05 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.394 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:8419 4.7 e54/40 ondisk, result = 0)", 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.431024+0000", 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8688781810000004, 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012172095000000001, 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431024+0000", 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431024+0000", 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431025+0000", 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3820000000000001e-06 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431032+0000", 2024-09-16T12:59:41.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4389999999999997e-06 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431054+0000", 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2008e-05 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431064+0000", 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-05 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443061+0000", 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011997017 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443069+0000", 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7309999999999993e-06 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443196+0000", 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012701800000000001 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8436 4.8 4:1430915d:::rbd_data.11b55eb65e78.0000000000000002:head [stat,truncate 14498372] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.493573+0000", 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.806329431, 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074513289999999996, 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8436 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493573+0000", 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493573+0000", 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493575+0000", 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4719999999999998e-06 2024-09-16T12:59:41.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493587+0000", 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1792e-05 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493627+0000", 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9743999999999998e-05 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493644+0000", 2024-09-16T12:59:41.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6796999999999999e-05 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.493778+0000", 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133934 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.494050+0000", 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027250100000000003 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.494428+0000", 2024-09-16T12:59:41.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037836199999999998 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.497239+0000", 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028101760000000002 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.500820+0000", 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003581608 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.500937+0000", 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011657799999999999 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.501024+0000", 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7365000000000001e-05 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8612 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.847921+0000", 2024-09-16T12:59:41.404 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4519810700000004, 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039491146999999997, 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8612 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847921+0000", 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847921+0000", 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847924+0000", 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0199999999999999e-06 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847936+0000", 2024-09-16T12:59:41.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1579999999999999e-05 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847968+0000", 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2033000000000003e-05 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847986+0000", 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7839e-05 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.848125+0000", 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013962800000000001 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.848277+0000", 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015126199999999999 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.856740+0000", 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084638479999999995 2024-09-16T12:59:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887229+0000", 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030488491 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887253+0000", 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4409e-05 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887347+0000", 2024-09-16T12:59:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3467999999999993e-05 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887412+0000", 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5568999999999995e-05 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:8612 4.7 e54/40 ondisk, result = 0)", 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.880870+0000", 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4190326070000001, 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063651840000000003, 2024-09-16T12:59:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880870+0000", 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880870+0000", 2024-09-16T12:59:41.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880871+0000", 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.463e-06 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880877+0000", 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0859999999999998e-06 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880898+0000", 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0979999999999999e-05 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.880907+0000", 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2920000000000001e-06 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887172+0000", 2024-09-16T12:59:41.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062649079999999996 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887211+0000", 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8695999999999997e-05 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887235+0000", 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3759000000000001e-05 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8851 4.8 4:1430915d:::rbd_data.11b55eb65e78.0000000000000002:head [delete] snapc e=[e] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:38.091084+0000", 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2088182349999999, 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057651029999999997, 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8851 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091084+0000", 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091084+0000", 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091087+0000", 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.847e-06 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091097+0000", 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0502999999999999e-05 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091129+0000", 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1513000000000001e-05 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091143+0000", 2024-09-16T12:59:41.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4535999999999999e-05 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091230+0000", 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6670999999999997e-05 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091356+0000", 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012561000000000001 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.091599+0000", 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000243685 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.093310+0000", 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017106689999999999 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.096773+0000", 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034631760000000001 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.096817+0000", 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3513000000000002e-05 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:38.096849+0000", 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2379999999999998e-05 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.420 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:41.422 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:41.422 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:41.415+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:41.523 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:41.519+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:41.528 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:41.528 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:41.528 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:41.528 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:41.537 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:41.624 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:41.615+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:41.641 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:41.641 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:41.641 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:41.641 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:41.641 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:41.650 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:41.725 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:41.719+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:41.754 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.963705768000001, 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089374989999999998, 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 11 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785975+0000", 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785978+0000", 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.373e-06 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.785989+0000", 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1596999999999999e-05 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786008+0000", 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8510999999999999e-05 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786016+0000", 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786102+0000", 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6339000000000001e-05 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.786113+0000", 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1386000000000001e-05 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.788774+0000", 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026609519999999998 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.788888+0000", 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113189 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.789043+0000", 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015570800000000001 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794705+0000", 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056619640000000002 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794770+0000", 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4789999999999995e-05 2024-09-16T12:59:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794853+0000", 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3441000000000003e-05 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:13.794913+0000", 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9157999999999998e-05 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:485 4.1f 4:fb389f96:::rbd_data.11b59d02cc38.0000000000000016:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.641497983000001, 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066273469999999996, 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 485 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108183+0000", 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108185+0000", 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3700000000000002e-06 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108195+0000", 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2839999999999995e-06 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108220+0000", 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5066999999999999e-05 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108229+0000", 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5019999999999998e-06 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108334+0000", 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105136 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.108571+0000", 2024-09-16T12:59:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236495 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.109793+0000", 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012221809999999999 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114733+0000", 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049399350000000003 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114745+0000", 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2228999999999999e-05 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114783+0000", 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7716999999999998e-05 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.114810+0000", 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7430999999999999e-05 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1248 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'11, mlcod=39'8)", 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.721638897999998, 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053616649999999998, 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028042+0000", 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028044+0000", 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5e-06 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028058+0000", 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4469e-05 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028091+0000", 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2517000000000001e-05 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028106+0000", 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5116000000000001e-05 2024-09-16T12:59:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028219+0000", 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113768 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.028268+0000", 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8745000000000002e-05 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033313+0000", 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050452869999999999 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.033404+0000", 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0263e-05 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:1250 4.18 e39/31 4:1f49b46c:::rbd_object_map.11b53140b009:head v 39'13, mlcod=39'8)", 2024-09-16T12:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.719317818, 2024-09-16T12:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061776180000000002, 2024-09-16T12:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030363+0000", 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030364+0000", 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.356e-06 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030375+0000", 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0053e-05 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030395+0000", 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0767e-05 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030404+0000", 2024-09-16T12:59:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4430000000000003e-06 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030492+0000", 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7842000000000004e-05 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.030514+0000", 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2819999999999998e-05 2024-09-16T12:59:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036453+0000", 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059381169999999997 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.036541+0000", 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8220000000000006e-05 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:1301 4.5 4:a692e91d:::rbd_data.11b53140b009.00000000000000c4:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.669280623999999, 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060951929999999996, 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1301 2024-09-16T12:59:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080400+0000", 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080402+0000", 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.75e-06 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080409+0000", 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2219999999999996e-06 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080429+0000", 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9527000000000001e-05 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080438+0000", 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7819999999999999e-06 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080503+0000", 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5438999999999995e-05 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080693+0000", 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190218 2024-09-16T12:59:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.080909+0000", 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021529000000000001 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.084016+0000", 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031074319999999998 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086352+0000", 2024-09-16T12:59:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023357690000000001 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086446+0000", 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4158000000000002e-05 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:19.086495+0000", 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9605999999999999e-05 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:3722 4.a 4:57b538f8:::rbd_header.11b5d5152bdd:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:22.903924+0000", 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.845756915999999, 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068689479999999997, 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3722 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903924+0000", 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903924+0000", 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903928+0000", 2024-09-16T12:59:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5099999999999999e-06 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903944+0000", 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6422999999999998e-05 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.903982+0000", 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8025999999999998e-05 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.904003+0000", 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0843e-05 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.904124+0000", 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012166 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.904135+0000", 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0703999999999999e-05 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.908409+0000", 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042740410000000001 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.908554+0000", 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014520900000000001 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.908771+0000", 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021657999999999999 2024-09-16T12:59:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.909753+0000", 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0009819430000000001 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.910682+0000", 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092942399999999998 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.910744+0000", 2024-09-16T12:59:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1796000000000003e-05 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:22.910793+0000", 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8789000000000003e-05 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:4074 4.a 4:57b538f8:::rbd_header.11b5d5152bdd:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:23.843397+0000", 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.906283929000001, 2024-09-16T12:59:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078647490000000007, 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4074 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843397+0000", 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843397+0000", 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843400+0000", 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5560000000000002e-06 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843409+0000", 2024-09-16T12:59:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6120000000000005e-06 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843434+0000", 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5125000000000001e-05 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843448+0000", 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3550000000000001e-05 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843527+0000", 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9359999999999999e-05 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.843534+0000", 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8220000000000001e-06 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.847907+0000", 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043724819999999996 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.848103+0000", 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019643899999999999 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.848776+0000", 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067343899999999996 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.850197+0000", 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014204269999999999 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.850995+0000", 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00079860300000000005 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.851155+0000", 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159479 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:23.851262+0000", 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010685500000000001 2024-09-16T12:59:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.385487207000001, 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T12:59:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T12:59:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T12:59:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T12:59:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5076 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.297769536000001, 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020443647999999998, 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5076 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451913+0000", 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451920+0000", 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5459999999999997e-06 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451938+0000", 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7581999999999999e-05 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451950+0000", 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2033e-05 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452042+0000", 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2533000000000003e-05 2024-09-16T12:59:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452251+0000", 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020887 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.456761+0000", 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045099959999999996 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472274+0000", 2024-09-16T12:59:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015512882 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472281+0000", 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3939999999999998e-06 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472324+0000", 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2389999999999999e-05 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472355+0000", 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1436000000000002e-05 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5232 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [write 26019337~28881] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.900074+0000", 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.849606872000001, 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058250639999999996, 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5232 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900074+0000", 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900074+0000", 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900077+0000", 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7700000000000002e-06 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900113+0000", 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6511999999999999e-05 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900156+0000", 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2483e-05 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900177+0000", 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1192999999999999e-05 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900300+0000", 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012320699999999999 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.900310+0000", 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6099999999999995e-06 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.903745+0000", 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034353640000000002 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.903815+0000", 2024-09-16T12:59:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9387999999999994e-05 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.903903+0000", 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8127999999999994e-05 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.904766+0000", 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086280999999999999 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.905841+0000", 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00107515 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.905877+0000", 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6720000000000001e-05 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.905899+0000", 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1729000000000002e-05 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5513 4.13 e46/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 46'80, mlcod=46'78)", 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.214469297000001, 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010685616, 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535213+0000", 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1060000000000001e-06 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535222+0000", 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0159999999999997e-06 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535240+0000", 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7926000000000001e-05 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535247+0000", 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5820000000000004e-06 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535345+0000", 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7705000000000001e-05 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535395+0000", 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9996e-05 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.545844+0000", 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010448687999999999 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.545897+0000", 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3597000000000002e-05 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5580 4.12 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.056646477999999, 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034176870999999998, 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5580 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693037+0000", 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0650000000000001e-06 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693045+0000", 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693070+0000", 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5341999999999999e-05 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693081+0000", 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0611000000000001e-05 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693182+0000", 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100437 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693309+0000", 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012741400000000001 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.703090+0000", 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097814800000000004 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726908+0000", 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023817680000000001 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727023+0000", 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011449400000000001 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727130+0000", 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010704099999999999 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727211+0000", 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1675000000000007e-05 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:5580 4.12 e46/40 ondisk, result = 0)", 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.718821+0000", 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.030859676, 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084041510000000003, 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718821+0000", 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718821+0000", 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718823+0000", 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8190000000000001e-06 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718832+0000", 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4549999999999995e-06 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718861+0000", 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9263000000000001e-05 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.718873+0000", 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1773999999999999e-05 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726990+0000", 2024-09-16T12:59:41.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081170989999999991 2024-09-16T12:59:41.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727007+0000", 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7416000000000001e-05 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727225+0000", 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021832500000000001 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:6817 4.1f 4:fa9bfe15:::rbd_object_map.11b58f77e1c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:32.612103+0000", 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1375774859999996, 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055784739999999999, 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 6817 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.612103+0000", 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.612103+0000", 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.612105+0000", 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.407e-06 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.612113+0000", 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2870000000000004e-06 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.612140+0000", 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6797000000000001e-05 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.612150+0000", 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0482000000000001e-05 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.612260+0000", 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109343 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.612444+0000", 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018472599999999999 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.613255+0000", 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081088799999999995 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.617406+0000", 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041505240000000001 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.617512+0000", 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106078 2024-09-16T12:59:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.617583+0000", 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1249999999999997e-05 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.617682+0000", 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8691999999999996e-05 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:6819 4.1f 4:fa9bfe15:::rbd_object_map.11b58f77e1c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:32.613426+0000", 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1362551530000005, 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070175949999999997, 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 6819 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.613426+0000", 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.613426+0000", 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.613427+0000", 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5e-06 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.613433+0000", 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4110000000000004e-06 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.613447+0000", 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4041e-05 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.613471+0000", 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4000000000000001e-05 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.615920+0000", 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024496269999999998 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.616163+0000", 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000242198 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.616732+0000", 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056986199999999997 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.620287+0000", 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035546620000000001 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.620346+0000", 2024-09-16T12:59:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9098999999999998e-05 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.620410+0000", 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3866000000000001e-05 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.620443+0000", 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3328999999999999e-05 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:6820 4.1f 4:fa9bfe15:::rbd_object_map.11b58f77e1c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:32.614851+0000", 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1348297580000004, 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061438669999999999, 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 6820 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.614851+0000", 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.614851+0000", 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.614852+0000", 2024-09-16T12:59:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1260000000000001e-06 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.614858+0000", 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1310000000000003e-06 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.614871+0000", 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2108e-05 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.614876+0000", 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9109999999999998e-06 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.617999+0000", 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031224389999999999 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.618071+0000", 2024-09-16T12:59:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2020000000000005e-05 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.618432+0000", 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000361159 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.620304+0000", 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018719310000000001 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.620929+0000", 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000624703 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.620968+0000", 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9279000000000001e-05 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.620995+0000", 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7059999999999998e-05 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:6821 4.1f 4:fa9bfe15:::rbd_object_map.11b58f77e1c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:32.615471+0000", 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1342100609999992, 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067940589999999999, 2024-09-16T12:59:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 6821 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.615471+0000", 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.615471+0000", 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.615472+0000", 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.331e-06 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.615480+0000", 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5610000000000001e-06 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.615496+0000", 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6101000000000001e-05 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.615504+0000", 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0120000000000005e-06 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.618948+0000", 2024-09-16T12:59:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003444375 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.619088+0000", 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013934799999999999 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.619766+0000", 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000678647 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.620861+0000", 2024-09-16T12:59:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010952519999999999 2024-09-16T12:59:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.622194+0000", 2024-09-16T12:59:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013326309999999999 2024-09-16T12:59:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.622235+0000", 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1325999999999997e-05 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:32.622265+0000", 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9475000000000002e-05 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:8686 4.e e54/31 4:77b45050:::rbd_object_map.11b55eb65e78:head v 54'415, mlcod=54'410)", 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.030615+0000", 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7190661580000004, 2024-09-16T12:59:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056130770000000002, 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030615+0000", 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030615+0000", 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030615+0000", 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9599999999999999e-07 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030623+0000", 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6480000000000003e-06 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030635+0000", 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1823e-05 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030642+0000", 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030728+0000", 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5432000000000005e-05 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030752+0000", 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4032999999999998e-05 2024-09-16T12:59:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.036181+0000", 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054294490000000003 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.036228+0000", 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6765999999999999e-05 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:8688 4.e e54/31 4:746c7c9d:::rbd_data.11b55eb65e78.0000000000000006:head v 54'416, mlcod=54'411)", 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.031596+0000", 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7180846580000004, 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053808809999999997, 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031596+0000", 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031596+0000", 2024-09-16T12:59:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031597+0000", 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9300000000000006e-07 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031604+0000", 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9229999999999996e-06 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031622+0000", 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8114999999999999e-05 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031631+0000", 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2640000000000008e-06 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031677+0000", 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6854999999999998e-05 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.031710+0000", 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2109e-05 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.036942+0000", 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052329439999999998 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.036977+0000", 2024-09-16T12:59:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4678000000000002e-05 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:8696 4.e e54/31 4:77b45050:::rbd_object_map.11b55eb65e78:head v 54'421, mlcod=54'417)", 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.037880+0000", 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7118005700000003, 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056432870000000003, 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.037880+0000", 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.037880+0000", 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.037881+0000", 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7100000000000011e-07 2024-09-16T12:59:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.037889+0000", 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1470000000000001e-06 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.037907+0000", 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7824000000000001e-05 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.037915+0000", 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.037971+0000", 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5435e-05 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.037993+0000", 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2585000000000001e-05 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.043476+0000", 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054831580000000001 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.043524+0000", 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7178000000000001e-05 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:41.867 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:41.869 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:41.870 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:41.863+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:41.970 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:41.967+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:41.974 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:41.975 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:41.975 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:41.975 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:41.984 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:42.071 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:42.067+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:42.129 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:42.129 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:42.129 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:42.130 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:42.130 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:42.138 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:42.172 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:42.163+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:42.241 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:42.241 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:42.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:42.241 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:42.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.241 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:42.241 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:42.241 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 56.686352005000003, 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:42.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T12:59:42.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.861525421000003, 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010809278, 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.248 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374549+0000", 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.037e-06 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374711+0000", 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016226599999999999 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374734+0000", 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2895999999999999e-05 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374746+0000", 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.221e-05 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.380942+0000", 2024-09-16T12:59:42.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061959390000000001 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381063+0000", 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121274 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381255+0000", 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019110500000000001 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384487+0000", 2024-09-16T12:59:42.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032327979999999998 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385296+0000", 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080888499999999996 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385332+0000", 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5774e-05 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385357+0000", 2024-09-16T12:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5094000000000001e-05 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.861314528999998, 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010976025, 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 45 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374760+0000", 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.13e-06 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374769+0000", 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5699999999999999e-06 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374819+0000", 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9882000000000001e-05 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374828+0000", 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382329+0000", 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075009009999999999 2024-09-16T12:59:42.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382403+0000", 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4208999999999994e-05 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382514+0000", 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111276 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385533+0000", 2024-09-16T12:59:42.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030190080000000001 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385564+0000", 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0926000000000002e-05 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385666+0000", 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101791 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385735+0000", 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8541999999999995e-05 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.861234957999997, 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.260 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.265 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.861187702000002, 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T12:59:42.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T12:59:42.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T12:59:42.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T12:59:42.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T12:59:42.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T12:59:42.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T12:59:42.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.860984766000001, 2024-09-16T12:59:42.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T12:59:42.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.860953504000001, 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T12:59:42.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T12:59:42.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T12:59:42.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.860917741999998, 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T12:59:42.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.860723112000002, 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T12:59:42.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T12:59:42.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T12:59:42.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.860676267000002, 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T12:59:42.295 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T12:59:42.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.316616226000001, 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010623719, 2024-09-16T12:59:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 69 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919458+0000", 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.474e-06 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919621+0000", 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016234800000000001 2024-09-16T12:59:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919641+0000", 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0336999999999999e-05 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919656+0000", 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4554e-05 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924273+0000", 2024-09-16T12:59:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046169610000000002 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924411+0000", 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138286 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924627+0000", 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021597399999999999 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928794+0000", 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004167268 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929983+0000", 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011884409999999999 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930039+0000", 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6363000000000002e-05 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930081+0000", 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1712999999999999e-05 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.316400096000002, 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T12:59:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.316346189000001, 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.316303884, 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T12:59:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T12:59:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T12:59:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.316145458999998, 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T12:59:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T12:59:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T12:59:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T12:59:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T12:59:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T12:59:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.316095978, 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T12:59:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T12:59:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.315965407999997, 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T12:59:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T12:59:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.315783895000003, 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.505256228, 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T12:59:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T12:59:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.505107273999997, 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T12:59:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T12:59:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T12:59:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.360 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.360 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.360 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:42.360 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:42.360 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:42.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:42.362 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:42.351+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:42.462 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:42.455+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:42.474 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:42.474 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:42.474 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:42.475 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:42.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:42.563 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:42.555+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:42.581 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:42.581 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:42.581 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:42.581 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:42.581 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:42.590 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:42.639 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:42.665 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:42.663+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:42.749 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:313 4.1e 4:7abe541a:::rbd_object_map.11b524d5182b.0000000000000004:head [writefull 0~35] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.939669139999999, 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053862449999999996, 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 313 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804330+0000", 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3309999999999998e-06 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804341+0000", 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1284000000000001e-05 2024-09-16T12:59:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804367+0000", 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6132999999999999e-05 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804380+0000", 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2686e-05 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804463+0000", 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3187000000000002e-05 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804472+0000", 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.149e-06 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.806749+0000", 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022770669999999998 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.806912+0000", 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016236500000000001 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.807071+0000", 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000158956 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.808001+0000", 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093026199999999995 2024-09-16T12:59:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809575+0000", 2024-09-16T12:59:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015741609999999999 2024-09-16T12:59:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809654+0000", 2024-09-16T12:59:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9028000000000003e-05 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809714+0000", 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9636000000000003e-05 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:486 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.637685273999999, 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077714180000000004, 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 486 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106314+0000", 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8329999999999998e-06 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106329+0000", 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4231e-05 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106369+0000", 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0074999999999998e-05 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106383+0000", 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4781e-05 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106471+0000", 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7472999999999994e-05 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106667+0000", 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000196503 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.107835+0000", 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011676670000000001 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.113820+0000", 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0059854069999999999 2024-09-16T12:59:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.113900+0000", 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0071e-05 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.114010+0000", 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109399 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.114083+0000", 2024-09-16T12:59:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2978000000000005e-05 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:636 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.271797687999999, 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027791489999999999, 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 636 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472202+0000", 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.01e-06 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472218+0000", 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5724000000000001e-05 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472261+0000", 2024-09-16T12:59:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2858999999999997e-05 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472277+0000", 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6297999999999999e-05 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472397+0000", 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012026199999999999 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472563+0000", 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016614900000000001 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.480681+0000", 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081177869999999996 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499671+0000", 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018990289 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499756+0000", 2024-09-16T12:59:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4511000000000005e-05 2024-09-16T12:59:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499858+0000", 2024-09-16T12:59:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102495 2024-09-16T12:59:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499991+0000", 2024-09-16T12:59:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132106 2024-09-16T12:59:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3577 4.1e 4:7a8a5a14:::rbd_data.11b5c378985b.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.643351236000001, 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027798529999999998, 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3577 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100649+0000", 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1769999999999998e-06 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100659+0000", 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0407e-05 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100689+0000", 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0290999999999998e-05 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100701+0000", 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1960999999999999e-05 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100800+0000", 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8350000000000005e-05 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.101027+0000", 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000227736 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.111830+0000", 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010802735000000001 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128285+0000", 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016454389999999999 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128338+0000", 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3112000000000002e-05 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128398+0000", 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0136999999999997e-05 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128444+0000", 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6233999999999998e-05 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:3577 4.1e e41/31 ondisk, result = 0)", 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.624185199999999, 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086403560000000001, 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119813+0000", 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4950000000000001e-06 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119822+0000", 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4210000000000001e-06 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119840+0000", 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7824000000000001e-05 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119848+0000", 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9750000000000007e-06 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128321+0000", 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084731100000000007 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128330+0000", 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619999999999998e-06 2024-09-16T12:59:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128452+0000", 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012226899999999999 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3672 4.16 4:6b60bac8:::rbd_header.11b5c378985b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.839314+0000", 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.904683194, 2024-09-16T12:59:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055860959999999996, 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3672 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839314+0000", 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839314+0000", 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839316+0000", 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2460000000000002e-06 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839326+0000", 2024-09-16T12:59:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0662000000000001e-05 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839352+0000", 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5086999999999999e-05 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839360+0000", 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9029999999999993e-06 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839453+0000", 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2719000000000005e-05 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839465+0000", 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1798e-05 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.843224+0000", 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037588460000000001 2024-09-16T12:59:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.843301+0000", 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7619000000000003e-05 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.843630+0000", 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032843999999999999 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844264+0000", 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063367499999999999 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844809+0000", 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00054593999999999997 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844844+0000", 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4603000000000001e-05 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844900+0000", 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5557999999999999e-05 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3894 4.1e 4:7bf418e8:::rbd_data.11b53140b009.0000000000000054:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.135227+0000", 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.608770081999999, 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049140659999999999, 2024-09-16T12:59:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3894 2024-09-16T12:59:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135227+0000", 2024-09-16T12:59:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135227+0000", 2024-09-16T12:59:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135228+0000", 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.15e-06 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135233+0000", 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0640000000000001e-06 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135250+0000", 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7334e-05 2024-09-16T12:59:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135261+0000", 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0407e-05 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.137529+0000", 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022684039999999999 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.138104+0000", 2024-09-16T12:59:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057520100000000001 2024-09-16T12:59:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140141+0000", 2024-09-16T12:59:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002036506 2024-09-16T12:59:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3932 4.6 4:61b26f91:::rbd_data.11b53140b009.000000000000007a:head [sparse-read 0~229376] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.607738210000001, 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046959480000000001, 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3932 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.37e-07 2024-09-16T12:59:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136261+0000", 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2000000000000001e-06 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136968+0000", 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070683599999999997 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136983+0000", 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4990999999999999e-05 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140450+0000", 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003467131 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140634+0000", 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018384400000000001 2024-09-16T12:59:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140954+0000", 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032030900000000001 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5066 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.321929123, 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019002768999999999, 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5066 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422070+0000", 2024-09-16T12:59:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-06 2024-09-16T12:59:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422082+0000", 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2224e-05 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422120+0000", 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7662e-05 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422133+0000", 2024-09-16T12:59:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3336000000000001e-05 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422296+0000", 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016319899999999999 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422539+0000", 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000243281 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.427723+0000", 2024-09-16T12:59:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051837580000000001 2024-09-16T12:59:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.440864+0000", 2024-09-16T12:59:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013140446 2024-09-16T12:59:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.440892+0000", 2024-09-16T12:59:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8628000000000001e-05 2024-09-16T12:59:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.441002+0000", 2024-09-16T12:59:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010946999999999999 2024-09-16T12:59:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.441070+0000", 2024-09-16T12:59:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8595000000000003e-05 2024-09-16T12:59:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5229 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.905774909, 2024-09-16T12:59:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049016400000000002, 2024-09-16T12:59:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5229 2024-09-16T12:59:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838225+0000", 2024-09-16T12:59:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0340000000000001e-06 2024-09-16T12:59:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838239+0000", 2024-09-16T12:59:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4004e-05 2024-09-16T12:59:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838280+0000", 2024-09-16T12:59:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0738000000000002e-05 2024-09-16T12:59:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838294+0000", 2024-09-16T12:59:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.482e-05 2024-09-16T12:59:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838415+0000", 2024-09-16T12:59:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012084 2024-09-16T12:59:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838570+0000", 2024-09-16T12:59:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000155162 2024-09-16T12:59:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.850428+0000", 2024-09-16T12:59:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011857826 2024-09-16T12:59:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887135+0000", 2024-09-16T12:59:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036706938000000001 2024-09-16T12:59:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887149+0000", 2024-09-16T12:59:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3421000000000001e-05 2024-09-16T12:59:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887202+0000", 2024-09-16T12:59:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3427000000000002e-05 2024-09-16T12:59:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887238+0000", 2024-09-16T12:59:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6189999999999997e-05 2024-09-16T12:59:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:5229 4.1e e45/31 ondisk, result = 0)", 2024-09-16T12:59:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.874885+0000", 2024-09-16T12:59:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.869111461999999, 2024-09-16T12:59:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012252984, 2024-09-16T12:59:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874885+0000", 2024-09-16T12:59:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874885+0000", 2024-09-16T12:59:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874887+0000", 2024-09-16T12:59:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.874e-06 2024-09-16T12:59:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874894+0000", 2024-09-16T12:59:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4529999999999999e-06 2024-09-16T12:59:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874917+0000", 2024-09-16T12:59:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3125999999999998e-05 2024-09-16T12:59:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874928+0000", 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1139e-05 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887109+0000", 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012181290000000001 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887125+0000", 2024-09-16T12:59:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5529e-05 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887138+0000", 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3573e-05 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5230 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.887669+0000", 2024-09-16T12:59:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.856328101000001, 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005832523, 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887669+0000", 2024-09-16T12:59:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887669+0000", 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887671+0000", 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2010000000000002e-06 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887681+0000", 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0465000000000001e-05 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887701+0000", 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9709999999999999e-05 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887709+0000", 2024-09-16T12:59:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6960000000000005e-06 2024-09-16T12:59:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887778+0000", 2024-09-16T12:59:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9363999999999995e-05 2024-09-16T12:59:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887784+0000", 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2129999999999996e-06 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.890175+0000", 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002390381 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.890275+0000", 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9994999999999994e-05 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.890554+0000", 2024-09-16T12:59:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027887099999999998 2024-09-16T12:59:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.892125+0000", 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015718539999999999 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.893435+0000", 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013094859999999999 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.893474+0000", 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8708999999999998e-05 2024-09-16T12:59:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.893501+0000", 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7577999999999999e-05 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5396 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.521953785999999, 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035598629999999999, 2024-09-16T12:59:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5396 2024-09-16T12:59:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222045+0000", 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9580000000000001e-06 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222055+0000", 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9310000000000003e-06 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222082+0000", 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6882000000000001e-05 2024-09-16T12:59:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222092+0000", 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8910000000000007e-06 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222166+0000", 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4437999999999994e-05 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222264+0000", 2024-09-16T12:59:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7748e-05 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.228318+0000", 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060540109999999998 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257411+0000", 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029093622999999999 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257500+0000", 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8341e-05 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257580+0000", 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257642+0000", 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1073000000000003e-05 2024-09-16T12:59:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:5396 4.1e e46/31 ondisk, result = 0)", 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.248841+0000", 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.495156028, 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088103839999999992, 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248841+0000", 2024-09-16T12:59:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248841+0000", 2024-09-16T12:59:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248842+0000", 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.012e-06 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248848+0000", 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.947e-06 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248865+0000", 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7588000000000002e-05 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248873+0000", 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257476+0000", 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086037709999999996 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257489+0000", 2024-09-16T12:59:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2334000000000001e-05 2024-09-16T12:59:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257651+0000", 2024-09-16T12:59:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016232799999999999 2024-09-16T12:59:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6557 4.6 4:62ff1677:::rbd_object_map.11b53fd98b75.000000000000000b:head [writefull 0~36] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:31.923520+0000", 2024-09-16T12:59:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.820476640000001, 2024-09-16T12:59:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053363129999999996, 2024-09-16T12:59:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6557 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923520+0000", 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923520+0000", 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923523+0000", 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923538+0000", 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4865000000000001e-05 2024-09-16T12:59:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923573+0000", 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5000999999999999e-05 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923587+0000", 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3505e-05 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923668+0000", 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260999999999993e-05 2024-09-16T12:59:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923675+0000", 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0199999999999997e-06 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.926586+0000", 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029107730000000002 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.926827+0000", 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000240898 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.927056+0000", 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022953200000000001 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928356+0000", 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012995999999999999 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928668+0000", 2024-09-16T12:59:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031215300000000002 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928792+0000", 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000124191 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928856+0000", 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4193e-05 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7047 4.1e 4:79fc4cc9:::rbd_data.11b58f77e1c4.0000000000000016:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.032558+0000", 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7114388169999994, 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045422789999999998, 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7047 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032558+0000", 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032558+0000", 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032559+0000", 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.375e-06 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032567+0000", 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3030000000000002e-06 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032630+0000", 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3369999999999998e-05 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032643+0000", 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.29e-05 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.033668+0000", 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001024655 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.033893+0000", 2024-09-16T12:59:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225109 2024-09-16T12:59:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.037100+0000", 2024-09-16T12:59:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032075670000000001 2024-09-16T12:59:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7086 4.16 4:6c47309f:::rbd_data.11b58f77e1c4.000000000000001a:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.043977+0000", 2024-09-16T12:59:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7000197630000002, 2024-09-16T12:59:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0058727320000000003, 2024-09-16T12:59:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7086 2024-09-16T12:59:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043977+0000", 2024-09-16T12:59:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043977+0000", 2024-09-16T12:59:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043978+0000", 2024-09-16T12:59:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1999999999999999e-07 2024-09-16T12:59:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043980+0000", 2024-09-16T12:59:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.661e-06 2024-09-16T12:59:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043988+0000", 2024-09-16T12:59:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9189999999999999e-06 2024-09-16T12:59:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043993+0000", 2024-09-16T12:59:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9350000000000002e-06 2024-09-16T12:59:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045633+0000", 2024-09-16T12:59:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016402400000000001 2024-09-16T12:59:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045747+0000", 2024-09-16T12:59:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113786 2024-09-16T12:59:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046061+0000", 2024-09-16T12:59:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031426000000000002 2024-09-16T12:59:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.047268+0000", 2024-09-16T12:59:42.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00120656 2024-09-16T12:59:42.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049764+0000", 2024-09-16T12:59:42.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024965180000000001 2024-09-16T12:59:42.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049813+0000", 2024-09-16T12:59:42.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8146999999999998e-05 2024-09-16T12:59:42.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049850+0000", 2024-09-16T12:59:42.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7085999999999997e-05 2024-09-16T12:59:42.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.871 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.871 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8151 4.1e 4:7e9eaa93:::rbd_object_map.11b59378797.000000000000000d:head [writefull 0~41] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:42.871 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:35.949190+0000", 2024-09-16T12:59:42.872 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.7948071560000001, 2024-09-16T12:59:42.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045078480000000001, 2024-09-16T12:59:42.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8151 2024-09-16T12:59:42.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.873 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.949190+0000", 2024-09-16T12:59:42.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.949190+0000", 2024-09-16T12:59:42.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.949192+0000", 2024-09-16T12:59:42.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.249e-06 2024-09-16T12:59:42.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.949214+0000", 2024-09-16T12:59:42.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1919000000000001e-05 2024-09-16T12:59:42.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.949239+0000", 2024-09-16T12:59:42.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5411000000000001e-05 2024-09-16T12:59:42.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.949249+0000", 2024-09-16T12:59:42.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6180000000000001e-06 2024-09-16T12:59:42.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.949313+0000", 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4455000000000005e-05 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.949318+0000", 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4349999999999999e-06 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.951697+0000", 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023798479999999999 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.951837+0000", 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139849 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.952001+0000", 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000163225 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.953008+0000", 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010070050000000001 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.953641+0000", 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063332700000000004 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.953673+0000", 2024-09-16T12:59:42.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2469999999999999e-05 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.953697+0000", 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4037e-05 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8437 4.1e 4:7d986d24:::rbd_data.11b55eb65e78.0000000000000003:head [stat,truncate 14155776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.491471+0000", 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.2525256520000001, 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050872399999999998, 2024-09-16T12:59:42.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8437 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491471+0000", 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491471+0000", 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491473+0000", 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0930000000000001e-06 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491484+0000", 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1032999999999999e-05 2024-09-16T12:59:42.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491513+0000", 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8575e-05 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491525+0000", 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2658e-05 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491634+0000", 2024-09-16T12:59:42.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010812899999999999 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491828+0000", 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019483200000000001 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.492191+0000", 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000362126 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:42.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.493735+0000", 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015445669999999999 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.496466+0000", 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027304740000000001 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.496515+0000", 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9364999999999999e-05 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.496558+0000", 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3387999999999999e-05 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8809 4.1e 4:7e9eaa93:::rbd_object_map.11b59378797.000000000000000d:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:38.022514+0000", 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.721482741, 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055643760000000002, 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:42.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:42.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:42.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:42.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8809 2024-09-16T12:59:42.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.888 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:42.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:42.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022514+0000", 2024-09-16T12:59:42.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:42.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022514+0000", 2024-09-16T12:59:42.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:42.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:42.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022517+0000", 2024-09-16T12:59:42.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1659999999999998e-06 2024-09-16T12:59:42.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:42.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022533+0000", 2024-09-16T12:59:42.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5477000000000001e-05 2024-09-16T12:59:42.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:42.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022572+0000", 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8995000000000002e-05 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022585+0000", 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3703000000000001e-05 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022737+0000", 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015165400000000001 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022755+0000", 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.804e-05 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.027473+0000", 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047179789999999997 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.027650+0000", 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017721600000000001 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:42.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:42.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:42.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.028078+0000", 2024-09-16T12:59:42.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042814600000000002 2024-09-16T12:59:42.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:42.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:42.894 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:42.895 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:42.896 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:42.896 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:42.896 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:42.896 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone12 2024-09-16T12:59:42.897 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:42.897 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:42.897 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:42.891+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:42.906 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:42.933 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:42.964 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:42.966 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:42.966 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:42.966 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:42.967 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:42.967 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:42.967 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:42.967 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:42.967 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:43.014 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T12:59:43.022 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:43.015+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:43.036 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:43.036 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:43.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:43.097 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:43.087+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:43.166 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:43.166 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:43.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:43.199 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:43.195+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:43.272 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:43.272 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:43.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:43.326 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:43.319+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:43.382 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:43.382 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:43.391 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:43.400 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:43.395+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:43.489 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:43.489 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:43.499 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:43.500 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:43.491+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:43.600 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:43.600 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:43.601 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:43.595+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:43.610 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:43.702 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:43.691+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:43.705 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:43.706 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:43.715 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:43.802 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:43.799+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:43.860 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:43.860 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:43.869 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:43.903 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:43.899+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:43.969 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:43.970 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:43.979 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:44.022 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:44.019+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:44.031 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T12:59:44.090 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:44.090 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:44.099 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:44.104 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:44.095+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:44.120 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-16T12:59:44.195 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:44.195 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:44.195 INFO:teuthology.orchestra.run.smithi042.stdout:1720 read 0x40fa670 thru 0x4104675 (0xa006 bytes) 2024-09-16T12:59:44.196 INFO:teuthology.orchestra.run.smithi042.stdout:1721 read 0x2ebef0b thru 0x2ec33fe (0x44f4 bytes) 2024-09-16T12:59:44.196 INFO:teuthology.orchestra.run.smithi042.stdout:1722 punch from 0x36a18f to 0x36e1a0, (0x4011 bytes) 2024-09-16T12:59:44.204 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:44.205 INFO:teuthology.orchestra.run.smithi042.stdout:1723 write 0x70362ed thru 0x703d37f (0x7093 bytes) 2024-09-16T12:59:44.206 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:44.199+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:44.212 INFO:teuthology.orchestra.run.smithi042.stdout:1724 read 0x3b157a6 thru 0x3b1afd0 (0x582b bytes) 2024-09-16T12:59:44.212 INFO:teuthology.orchestra.run.smithi042.stdout:1725 write 0xdf909b thru 0xe013e4 (0x834a bytes) 2024-09-16T12:59:44.222 INFO:teuthology.orchestra.run.smithi042.stdout:1726 read 0x494157b thru 0x494d14a (0xbbd0 bytes) 2024-09-16T12:59:44.222 INFO:teuthology.orchestra.run.smithi042.stdout:1727 punch from 0x105f08c to 0x106c08d, (0xd001 bytes) 2024-09-16T12:59:44.239 INFO:teuthology.orchestra.run.smithi042.stdout:1728 write 0xecb413c thru 0xecba3c4 (0x6289 bytes) 2024-09-16T12:59:44.244 INFO:teuthology.orchestra.run.smithi042.stdout:1729 punch from 0x6308705 to 0x6309250, (0xb4b bytes) 2024-09-16T12:59:44.245 INFO:teuthology.orchestra.run.smithi042.stdout:1731 write 0xd02ed41 thru 0xd03e953 (0xfc13 bytes) 2024-09-16T12:59:44.251 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:44.251 INFO:teuthology.orchestra.run.smithi042.stdout:1734 read 0xdc2d695 thru 0xdc2db1d (0x489 bytes) 2024-09-16T12:59:44.251 INFO:teuthology.orchestra.run.smithi042.stdout:1735 read 0x9151687 thru 0x915845a (0x6dd4 bytes) 2024-09-16T12:59:44.251 INFO:teuthology.orchestra.run.smithi042.stdout:1736 write 0xbbe6c8c thru 0xbbeea0d (0x7d82 bytes) 2024-09-16T12:59:44.256 INFO:teuthology.orchestra.run.smithi042.stdout:1737 writesame 0x2a76def thru 0x2a78a9d data_size 0x1(0x1caf bytes) 2024-09-16T12:59:44.268 INFO:teuthology.orchestra.run.smithi042.stdout:1738 writesame 0x4d869c7 thru 0x4d91240 data_size 0x5f(0xa87a bytes) 2024-09-16T12:59:44.276 INFO:teuthology.orchestra.run.smithi042.stdout:1739 read 0x85bde60 thru 0x85cd573 (0xf714 bytes) 2024-09-16T12:59:44.277 INFO:teuthology.orchestra.run.smithi042.stdout:1740 write 0xbe758e1 thru 0xbe762af (0x9cf bytes) 2024-09-16T12:59:44.283 INFO:teuthology.orchestra.run.smithi042.stdout:1744 punch from 0x6a0427a to 0x6a0ffb3, (0xbd39 bytes) 2024-09-16T12:59:44.283 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:44.283 INFO:teuthology.orchestra.run.smithi042.stdout:1746 read 0x2e57d8d thru 0x2e5db5b (0x5dcf bytes) 2024-09-16T12:59:44.284 INFO:teuthology.orchestra.run.smithi042.stdout:1747 read 0x6eec98 thru 0x6fb052 (0xc3bb bytes) 2024-09-16T12:59:44.305 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:44.305 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:44.306 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:44.299+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:44.308 INFO:teuthology.orchestra.run.smithi042.stdout:1748 trunc from 0xecba3c5 to 0x38b2b05 2024-09-16T12:59:44.315 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:44.343 INFO:teuthology.orchestra.run.smithi042.stdout:1749 read 0x2124e0d thru 0x21320f0 (0xd2e4 bytes) 2024-09-16T12:59:44.348 INFO:teuthology.orchestra.run.smithi042.stdout:1750 trunc from 0x38b2b05 to 0x5532b91 2024-09-16T12:59:44.380 INFO:teuthology.orchestra.run.smithi042.stdout:1752 writesame 0xb4925d9 thru 0xb4936bb data_size 0xb(0x10e3 bytes) 2024-09-16T12:59:44.396 INFO:teuthology.orchestra.run.smithi042.stdout:1753 trunc from 0xb4936bc to 0x34abd3d 2024-09-16T12:59:44.407 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:44.403+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:44.416 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:44.417 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:44.426 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:44.439 INFO:teuthology.orchestra.run.smithi042.stdout:1754 writesame 0xa0a87c9 thru 0xa0ac17c data_size 0x6(0x39b4 bytes) 2024-09-16T12:59:44.439 INFO:teuthology.orchestra.run.smithi042.stdout:1755 read 0x97a33df thru 0x97a98e0 (0x6502 bytes) 2024-09-16T12:59:44.441 INFO:teuthology.orchestra.run.smithi042.stdout:1756 trunc from 0xa0ac17d to 0xb725e02 2024-09-16T12:59:44.451 INFO:teuthology.orchestra.run.smithi042.stdout:1757 writesame 0x3139d07 thru 0x3139e4b data_size 0x1(0x145 bytes) 2024-09-16T12:59:44.466 INFO:teuthology.orchestra.run.smithi042.stdout:1760 punch from 0x565319a to 0x5654f5e, (0x1dc4 bytes) 2024-09-16T12:59:44.466 INFO:teuthology.orchestra.run.smithi042.stdout:1761 read 0x2f31b7b thru 0x2f3bece (0xa354 bytes) 2024-09-16T12:59:44.468 INFO:teuthology.orchestra.run.smithi042.stdout:1762 trunc from 0xb725e02 to 0xc2cf14d 2024-09-16T12:59:44.478 INFO:teuthology.orchestra.run.smithi042.stdout:1763 write 0x2304b79 thru 0x230e408 (0x9890 bytes) 2024-09-16T12:59:44.481 INFO:teuthology.orchestra.run.smithi042.stdout:1765 writesame 0x8b81e44 thru 0x8b875d7 data_size 0x3b(0x5794 bytes) 2024-09-16T12:59:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:1766 write 0x6cdafa1 thru 0x6ceac7d (0xfcdd bytes) 2024-09-16T12:59:44.494 INFO:teuthology.orchestra.run.smithi042.stdout:1768 clone 14 order 24 su 8388608 sc 4 2024-09-16T12:59:44.508 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:44.503+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:44.527 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:44.527 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:44.536 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:44.609 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:44.603+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:44.639 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:44.639 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:44.648 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:44.710 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:44.707+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:44.748 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:44.748 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:44.757 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:44.811 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:44.803+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:44.903 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:44.904 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:44.912 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:44.913 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:44.903+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:44.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:45.010 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:45.010 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:45.010 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T12:59:45.010 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T12:59:45.010 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-16T12:59:45.010 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T12:59:45.010 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T12:59:45.011 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-16T12:59:45.012 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:45.007+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:45.019 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:45.113 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:45.107+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:45.117 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:45.117 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T12:59:45.117 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone13 from 0xc2cf14d (overlap 0x34abd3d) to 0x2f6771d 2024-09-16T12:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:45.164 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:45.173 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:45.173 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:45.183 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:45.195 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:45.215 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:45.207+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:45.221 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:45.243 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:45.266 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:45.282 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:45.283 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:45.284 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-16T12:59:45.292 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:45.298 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:45.314 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:45.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:45.342 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:45.339+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:45.408 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.854489874000002, 2024-09-16T12:59:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T12:59:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T12:59:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T12:59:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.029663290000002, 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010809278, 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374549+0000", 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.037e-06 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374711+0000", 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016226599999999999 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374734+0000", 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2895999999999999e-05 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374746+0000", 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.221e-05 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.380942+0000", 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061959390000000001 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381063+0000", 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121274 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381255+0000", 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019110500000000001 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384487+0000", 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032327979999999998 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385296+0000", 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080888499999999996 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385332+0000", 2024-09-16T12:59:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5774e-05 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385357+0000", 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5094000000000001e-05 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.029452397999997, 2024-09-16T12:59:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010976025, 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 45 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374760+0000", 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.13e-06 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374769+0000", 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5699999999999999e-06 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374819+0000", 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9882000000000001e-05 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374828+0000", 2024-09-16T12:59:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382329+0000", 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075009009999999999 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382403+0000", 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4208999999999994e-05 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382514+0000", 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111276 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385533+0000", 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030190080000000001 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385564+0000", 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0926000000000002e-05 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385666+0000", 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101791 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385735+0000", 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8541999999999995e-05 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.029372827000003, 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T12:59:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T12:59:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T12:59:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T12:59:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T12:59:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.029325571000001, 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T12:59:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T12:59:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T12:59:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T12:59:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T12:59:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T12:59:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T12:59:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T12:59:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T12:59:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.029122635, 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T12:59:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T12:59:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.029091373, 2024-09-16T12:59:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T12:59:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T12:59:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.029055610999997, 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T12:59:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T12:59:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T12:59:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T12:59:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T12:59:45.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T12:59:45.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T12:59:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.028860981000001, 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:45.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T12:59:45.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T12:59:45.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T12:59:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T12:59:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T12:59:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T12:59:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T12:59:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T12:59:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T12:59:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T12:59:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T12:59:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T12:59:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T12:59:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T12:59:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T12:59:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T12:59:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T12:59:45.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T12:59:45.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T12:59:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T12:59:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T12:59:45.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T12:59:45.474 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.474 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.474 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.475 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:45.475 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:45.475 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.028814136000001, 2024-09-16T12:59:45.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T12:59:45.475 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T12:59:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T12:59:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T12:59:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T12:59:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T12:59:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.484754095, 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010623719, 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 69 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919458+0000", 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.474e-06 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919621+0000", 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016234800000000001 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919641+0000", 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0336999999999999e-05 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919656+0000", 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4554e-05 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924273+0000", 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046169610000000002 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924411+0000", 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138286 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924627+0000", 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021597399999999999 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928794+0000", 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004167268 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929983+0000", 2024-09-16T12:59:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011884409999999999 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930039+0000", 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6363000000000002e-05 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930081+0000", 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1712999999999999e-05 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.484537965000001, 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T12:59:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T12:59:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.484484058, 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T12:59:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.484441752999999, 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T12:59:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T12:59:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.484283327999997, 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T12:59:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T12:59:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T12:59:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.484233846999999, 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T12:59:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T12:59:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.484103277000003, 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T12:59:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T12:59:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.483921764000002, 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T12:59:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.673394096999999, 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T12:59:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T12:59:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.673245143000003, 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T12:59:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T12:59:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:45.543 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:45.545 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:45.539+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:45.545 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:45.624 INFO:teuthology.orchestra.run.smithi042.stdout:1769 trunc from 0xc2cf14d to 0xe5e5b57 2024-09-16T12:59:45.645 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:45.639+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:45.646 INFO:teuthology.orchestra.run.smithi042.stdout:1770 writesame 0x53b30f6 thru 0x53b495e data_size 0x1(0x1869 bytes) 2024-09-16T12:59:45.651 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:45.652 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:45.652 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:45.652 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:45.658 INFO:teuthology.orchestra.run.smithi042.stdout:1773 write 0xa1c7f8 thru 0xa243c8 (0x7bd1 bytes) 2024-09-16T12:59:45.662 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout:1774 write 0xb899888 thru 0xb89f665 (0x5dde bytes) 2024-09-16T12:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout:1775 writesame 0xc9b4290 thru 0xc9bc361 data_size 0xb(0x80d2 bytes) 2024-09-16T12:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout:1776 writesame 0x25c82b4 thru 0x25d1122 data_size 0x1(0x8e6f bytes) 2024-09-16T12:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout:1778 trunc from 0xe5e5b57 to 0xa721499 2024-09-16T12:59:45.746 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:45.739+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout:1782 trunc from 0xa721499 to 0xa7f98bf 2024-09-16T12:59:45.768 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:45.768 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:45.768 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:45.769 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:45.769 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:45.779 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:45.781 INFO:teuthology.orchestra.run.smithi042.stdout:1783 write 0xbb5f9e8 thru 0xbb65fdf (0x65f8 bytes) 2024-09-16T12:59:45.802 INFO:teuthology.orchestra.run.smithi042.stdout:1784 write 0xd6ac3c3 thru 0xd6b5b28 (0x9766 bytes) 2024-09-16T12:59:45.814 INFO:teuthology.orchestra.run.smithi042.stdout:1785 trunc from 0xd6b5b29 to 0xe78a9ac 2024-09-16T12:59:45.828 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:45.828 INFO:teuthology.orchestra.run.smithi042.stdout:1789 write 0x655235 thru 0x6624e6 (0xd2b2 bytes) 2024-09-16T12:59:45.832 INFO:teuthology.orchestra.run.smithi042.stdout:1790 read 0x2da6915 thru 0x2db41a8 (0xd894 bytes) 2024-09-16T12:59:45.833 INFO:teuthology.orchestra.run.smithi042.stdout:1792 punch from 0xcaeb117 to 0xcaefc1e, (0x4b07 bytes) 2024-09-16T12:59:45.833 INFO:teuthology.orchestra.run.smithi042.stdout:1793 write 0xc9c97c5 thru 0xc9d1021 (0x785d bytes) 2024-09-16T12:59:45.840 INFO:teuthology.orchestra.run.smithi042.stdout:1794 read 0x908e57b thru 0x909a36a (0xbdf0 bytes) 2024-09-16T12:59:45.842 INFO:teuthology.orchestra.run.smithi042.stdout:1795 punch from 0x337c2e3 to 0x33874ed, (0xb20a bytes) 2024-09-16T12:59:45.842 INFO:teuthology.orchestra.run.smithi042.stdout:1797 writesame 0x898b4b thru 0x8a5663 data_size 0x6d(0xcb19 bytes) 2024-09-16T12:59:45.845 INFO:teuthology.orchestra.run.smithi042.stdout:1800 write 0x28b3646 thru 0x28bb6cb (0x8086 bytes) 2024-09-16T12:59:45.846 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:45.839+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:45.848 INFO:teuthology.orchestra.run.smithi042.stdout:1801 read 0x86f91fa thru 0x86fc500 (0x3307 bytes) 2024-09-16T12:59:45.849 INFO:teuthology.orchestra.run.smithi042.stdout:1802 punch from 0x72a93a8 to 0x72b05a0, (0x71f8 bytes) 2024-09-16T12:59:45.849 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:45.849 INFO:teuthology.orchestra.run.smithi042.stdout:1804 write 0xb0cf601 thru 0xb0dd338 (0xdd38 bytes) 2024-09-16T12:59:45.853 INFO:teuthology.orchestra.run.smithi042.stdout:1805 write 0x86018a5 thru 0x860482d (0x2f89 bytes) 2024-09-16T12:59:45.856 INFO:teuthology.orchestra.run.smithi042.stdout:1806 write 0x67d1cd8 thru 0x67df4a2 (0xd7cb bytes) 2024-09-16T12:59:45.865 INFO:teuthology.orchestra.run.smithi042.stdout:1807 writesame 0x68cd87a thru 0x68d6c71 data_size 0x14(0x93f8 bytes) 2024-09-16T12:59:45.876 INFO:teuthology.orchestra.run.smithi042.stdout:1808 punch from 0x62c03f2 to 0x62c820f, (0x7e1d bytes) 2024-09-16T12:59:45.876 INFO:teuthology.orchestra.run.smithi042.stdout:1811 read 0xe358fd2 thru 0xe35e360 (0x538f bytes) 2024-09-16T12:59:45.876 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:45.876 INFO:teuthology.orchestra.run.smithi042.stdout:1813 read 0x599ab83 thru 0x59a53e5 (0xa863 bytes) 2024-09-16T12:59:45.876 INFO:teuthology.orchestra.run.smithi042.stdout:1814 write 0x646dc84 thru 0x6472d71 (0x50ee bytes) 2024-09-16T12:59:45.879 INFO:teuthology.orchestra.run.smithi042.stdout:1815 write 0x57bca2a thru 0x57bf01f (0x25f6 bytes) 2024-09-16T12:59:45.890 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:313 4.1e 4:7abe541a:::rbd_object_map.11b524d5182b.0000000000000004:head [writefull 0~35] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.077248190999999, 2024-09-16T12:59:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053862449999999996, 2024-09-16T12:59:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 313 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804328+0000", 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804330+0000", 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3309999999999998e-06 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804341+0000", 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1284000000000001e-05 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804367+0000", 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6132999999999999e-05 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804380+0000", 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2686e-05 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804463+0000", 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3187000000000002e-05 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.804472+0000", 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.149e-06 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.806749+0000", 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022770669999999998 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.806912+0000", 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016236500000000001 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.807071+0000", 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000158956 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.808001+0000", 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093026199999999995 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809575+0000", 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015741609999999999 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809654+0000", 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9028000000000003e-05 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.809714+0000", 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9636000000000003e-05 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:486 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.775264324999998, 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077714180000000004, 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 486 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106311+0000", 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106314+0000", 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8329999999999998e-06 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106329+0000", 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4231e-05 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106369+0000", 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0074999999999998e-05 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106383+0000", 2024-09-16T12:59:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4781e-05 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106471+0000", 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7472999999999994e-05 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.106667+0000", 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000196503 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.107835+0000", 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011676670000000001 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.113820+0000", 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0059854069999999999 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.113900+0000", 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0071e-05 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.114010+0000", 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109399 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.114083+0000", 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2978000000000005e-05 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:636 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.409376738999999, 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027791489999999999, 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 636 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472202+0000", 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.01e-06 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472218+0000", 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5724000000000001e-05 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472261+0000", 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2858999999999997e-05 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472277+0000", 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6297999999999999e-05 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472397+0000", 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012026199999999999 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472563+0000", 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016614900000000001 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.480681+0000", 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081177869999999996 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499671+0000", 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018990289 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499756+0000", 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4511000000000005e-05 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499858+0000", 2024-09-16T12:59:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102495 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499991+0000", 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132106 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3577 4.1e 4:7a8a5a14:::rbd_data.11b5c378985b.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.780930287, 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027798529999999998, 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3577 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100649+0000", 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1769999999999998e-06 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100659+0000", 2024-09-16T12:59:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0407e-05 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100689+0000", 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0290999999999998e-05 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100701+0000", 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1960999999999999e-05 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100800+0000", 2024-09-16T12:59:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8350000000000005e-05 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.101027+0000", 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000227736 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.111830+0000", 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010802735000000001 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128285+0000", 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016454389999999999 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128338+0000", 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3112000000000002e-05 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128398+0000", 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0136999999999997e-05 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128444+0000", 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6233999999999998e-05 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:3577 4.1e e41/31 ondisk, result = 0)", 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.761764250999999, 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086403560000000001, 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119812+0000", 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119813+0000", 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4950000000000001e-06 2024-09-16T12:59:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119822+0000", 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4210000000000001e-06 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119840+0000", 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7824000000000001e-05 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.119848+0000", 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9750000000000007e-06 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128321+0000", 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084731100000000007 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128330+0000", 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619999999999998e-06 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128452+0000", 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012226899999999999 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3672 4.16 4:6b60bac8:::rbd_header.11b5c378985b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.839314+0000", 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.042262245, 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055860959999999996, 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3672 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839314+0000", 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839314+0000", 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839316+0000", 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2460000000000002e-06 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839326+0000", 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0662000000000001e-05 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839352+0000", 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5086999999999999e-05 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839360+0000", 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9029999999999993e-06 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839453+0000", 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2719000000000005e-05 2024-09-16T12:59:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.839465+0000", 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1798e-05 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.843224+0000", 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037588460000000001 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.843301+0000", 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7619000000000003e-05 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.843630+0000", 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032843999999999999 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844264+0000", 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063367499999999999 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844809+0000", 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00054593999999999997 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844844+0000", 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4603000000000001e-05 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.844900+0000", 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5557999999999999e-05 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3894 4.1e 4:7bf418e8:::rbd_data.11b53140b009.0000000000000054:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.135227+0000", 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.746349132999999, 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049140659999999999, 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3894 2024-09-16T12:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135227+0000", 2024-09-16T12:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135227+0000", 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135228+0000", 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.15e-06 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135233+0000", 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0640000000000001e-06 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135250+0000", 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7334e-05 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.135261+0000", 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0407e-05 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.137529+0000", 2024-09-16T12:59:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022684039999999999 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.138104+0000", 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057520100000000001 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140141+0000", 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002036506 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3932 4.6 4:61b26f91:::rbd_data.11b53140b009.000000000000007a:head [sparse-read 0~229376] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.745317261, 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046959480000000001, 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3932 2024-09-16T12:59:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136259+0000", 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.37e-07 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136261+0000", 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2000000000000001e-06 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136968+0000", 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070683599999999997 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.136983+0000", 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4990999999999999e-05 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140450+0000", 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003467131 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140634+0000", 2024-09-16T12:59:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018384400000000001 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.140954+0000", 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032030900000000001 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5066 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.459508174, 2024-09-16T12:59:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019002768999999999, 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5066 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422070+0000", 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-06 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422082+0000", 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2224e-05 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422120+0000", 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7662e-05 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422133+0000", 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3336000000000001e-05 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422296+0000", 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016319899999999999 2024-09-16T12:59:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422539+0000", 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000243281 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.427723+0000", 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051837580000000001 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.440864+0000", 2024-09-16T12:59:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013140446 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.440892+0000", 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8628000000000001e-05 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.441002+0000", 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010946999999999999 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.441070+0000", 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8595000000000003e-05 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5229 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.043353960000001, 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049016400000000002, 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5229 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838225+0000", 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0340000000000001e-06 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838239+0000", 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4004e-05 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838280+0000", 2024-09-16T12:59:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0738000000000002e-05 2024-09-16T12:59:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838294+0000", 2024-09-16T12:59:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.482e-05 2024-09-16T12:59:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838415+0000", 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012084 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838570+0000", 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000155162 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.850428+0000", 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011857826 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887135+0000", 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036706938000000001 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887149+0000", 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3421000000000001e-05 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887202+0000", 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3427000000000002e-05 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887238+0000", 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6189999999999997e-05 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:5229 4.1e e45/31 ondisk, result = 0)", 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.874885+0000", 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.006690512999999, 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012252984, 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874885+0000", 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874885+0000", 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874887+0000", 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.874e-06 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874894+0000", 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4529999999999999e-06 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874917+0000", 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3125999999999998e-05 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.874928+0000", 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1139e-05 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887109+0000", 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012181290000000001 2024-09-16T12:59:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887125+0000", 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5529e-05 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887138+0000", 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3573e-05 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5230 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.887669+0000", 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.993907151999998, 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005832523, 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887669+0000", 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887669+0000", 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887671+0000", 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2010000000000002e-06 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887681+0000", 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0465000000000001e-05 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887701+0000", 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9709999999999999e-05 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887709+0000", 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6960000000000005e-06 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887778+0000", 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9363999999999995e-05 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887784+0000", 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2129999999999996e-06 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.890175+0000", 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002390381 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.890275+0000", 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9994999999999994e-05 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.890554+0000", 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027887099999999998 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.892125+0000", 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015718539999999999 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.893435+0000", 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013094859999999999 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.893474+0000", 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8708999999999998e-05 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.893501+0000", 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7577999999999999e-05 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5396 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.659532837, 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035598629999999999, 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5396 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222045+0000", 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9580000000000001e-06 2024-09-16T12:59:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222055+0000", 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9310000000000003e-06 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222082+0000", 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6882000000000001e-05 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222092+0000", 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8910000000000007e-06 2024-09-16T12:59:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222166+0000", 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4437999999999994e-05 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222264+0000", 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7748e-05 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.228318+0000", 2024-09-16T12:59:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060540109999999998 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257411+0000", 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029093622999999999 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257500+0000", 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8341e-05 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257580+0000", 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257642+0000", 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1073000000000003e-05 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:5396 4.1e e46/31 ondisk, result = 0)", 2024-09-16T12:59:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.248841+0000", 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.632735079, 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088103839999999992, 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248841+0000", 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248841+0000", 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248842+0000", 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.012e-06 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248848+0000", 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.947e-06 2024-09-16T12:59:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248865+0000", 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7588000000000002e-05 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.248873+0000", 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257476+0000", 2024-09-16T12:59:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086037709999999996 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257489+0000", 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2334000000000001e-05 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257651+0000", 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016232799999999999 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6557 4.6 4:62ff1677:::rbd_object_map.11b53fd98b75.000000000000000b:head [writefull 0~36] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:31.923520+0000", 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.958055691, 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053363129999999996, 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6557 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923520+0000", 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923520+0000", 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923523+0000", 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923538+0000", 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4865000000000001e-05 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923573+0000", 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5000999999999999e-05 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923587+0000", 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3505e-05 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923668+0000", 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260999999999993e-05 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.923675+0000", 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0199999999999997e-06 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.926586+0000", 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029107730000000002 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.926827+0000", 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000240898 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.927056+0000", 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022953200000000001 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928356+0000", 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012995999999999999 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928668+0000", 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031215300000000002 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928792+0000", 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000124191 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:31.928856+0000", 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4193e-05 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7047 4.1e 4:79fc4cc9:::rbd_data.11b58f77e1c4.0000000000000016:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.032558+0000", 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.849017868000001, 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045422789999999998, 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7047 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032558+0000", 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032558+0000", 2024-09-16T12:59:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032559+0000", 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.375e-06 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032567+0000", 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3030000000000002e-06 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032630+0000", 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3369999999999998e-05 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.032643+0000", 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.29e-05 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.033668+0000", 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001024655 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.033893+0000", 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225109 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.037100+0000", 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032075670000000001 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7086 4.16 4:6c47309f:::rbd_data.11b58f77e1c4.000000000000001a:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.043977+0000", 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.837598814, 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0058727320000000003, 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7086 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043977+0000", 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043977+0000", 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043978+0000", 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1999999999999999e-07 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043980+0000", 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.661e-06 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043988+0000", 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9189999999999999e-06 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043993+0000", 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9350000000000002e-06 2024-09-16T12:59:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045633+0000", 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016402400000000001 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045747+0000", 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113786 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046061+0000", 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031426000000000002 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.047268+0000", 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00120656 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049764+0000", 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024965180000000001 2024-09-16T12:59:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049813+0000", 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8146999999999998e-05 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049850+0000", 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7085999999999997e-05 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8437 4.1e 4:7d986d24:::rbd_data.11b55eb65e78.0000000000000003:head [stat,truncate 14155776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.491471+0000", 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3901047030000004, 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050872399999999998, 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8437 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491471+0000", 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491471+0000", 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491473+0000", 2024-09-16T12:59:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0930000000000001e-06 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491484+0000", 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1032999999999999e-05 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491513+0000", 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8575e-05 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491525+0000", 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2658e-05 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491634+0000", 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010812899999999999 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.491828+0000", 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019483200000000001 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.492191+0000", 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000362126 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.493735+0000", 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015445669999999999 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.496466+0000", 2024-09-16T12:59:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027304740000000001 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.496515+0000", 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9364999999999999e-05 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.496558+0000", 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3387999999999999e-05 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8809 4.1e 4:7e9eaa93:::rbd_object_map.11b59378797.000000000000000d:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:38.022514+0000", 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.8590617920000003, 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055643760000000002, 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8809 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022514+0000", 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022514+0000", 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022517+0000", 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1659999999999998e-06 2024-09-16T12:59:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022533+0000", 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5477000000000001e-05 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022572+0000", 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8995000000000002e-05 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022585+0000", 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3703000000000001e-05 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022737+0000", 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015165400000000001 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.022755+0000", 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.804e-05 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.027473+0000", 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047179789999999997 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.027650+0000", 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017721600000000001 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.028078+0000", 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042814600000000002 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:12973 4.1e 4:7cca8bfd:::rbd_data.11b571860f2a.0000000000000000:head [stat,truncate 4194304] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:45.142896+0000", 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.73867988600000001, 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054679180000000004, 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 12973 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.142896+0000", 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.142896+0000", 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.142897+0000", 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5239999999999999e-06 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.142906+0000", 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2139999999999996e-06 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.142928+0000", 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2141e-05 2024-09-16T12:59:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.142939+0000", 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1513e-05 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.143055+0000", 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115266 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.143289+0000", 2024-09-16T12:59:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023403000000000001 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.143734+0000", 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044547 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.146912+0000", 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031777630000000001 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.148214+0000", 2024-09-16T12:59:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001302522 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.148306+0000", 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1558000000000007e-05 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.148364+0000", 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7917000000000001e-05 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.001 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.001 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:46.002 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:46.003 INFO:teuthology.orchestra.run.smithi042.stdout:1817 trunc from 0xe78a9ac to 0x2b05b59 2024-09-16T12:59:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:1818 read 0x145fa1a thru 0x1460779 (0xd60 bytes) 2024-09-16T12:59:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:1821 writesame 0x7af3113 thru 0x7afe7af data_size 0x1(0xb69d bytes) 2024-09-16T12:59:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:1823 punch from 0x21d4be7 to 0x21da251, (0x566a bytes) 2024-09-16T12:59:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:1825 writesame 0x5982ede thru 0x598d82b data_size 0xd(0xa94e bytes) 2024-09-16T12:59:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:1826 punch from 0x680071c to 0x6803183, (0x2a67 bytes) 2024-09-16T12:59:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:1829 writesame 0xc11564e thru 0xc11d109 data_size 0xa(0x7abc bytes) 2024-09-16T12:59:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:1832 write 0x844334b thru 0x8452585 (0xf23b bytes) 2024-09-16T12:59:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:1834 write 0x94750b8 thru 0x9478d69 (0x3cb2 bytes) 2024-09-16T12:59:46.004 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:45.995+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:46.005 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:46.005 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:46.005 INFO:teuthology.orchestra.run.smithi042.stdout:1838 read 0xa26feed thru 0xa27c477 (0xc58b bytes) 2024-09-16T12:59:46.005 INFO:teuthology.orchestra.run.smithi042.stdout:1839 punch from 0xb9e5b01 to 0xb9f0f66, (0xb465 bytes) 2024-09-16T12:59:46.022 INFO:teuthology.orchestra.run.smithi042.stdout:1840 write 0xcdf431c thru 0xcdfeb2e (0xa813 bytes) 2024-09-16T12:59:46.028 INFO:teuthology.orchestra.run.smithi042.stdout:1841 punch from 0x5e7edc2 to 0x5e8d00d, (0xe24b bytes) 2024-09-16T12:59:46.028 INFO:teuthology.orchestra.run.smithi042.stdout:1842 write 0x2882615 thru 0x28892d2 (0x6cbe bytes) 2024-09-16T12:59:46.030 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:46.042 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T12:59:46.045 INFO:teuthology.orchestra.run.smithi042.stdout:1846 trunc from 0xcdfeb2f to 0x5927686 2024-09-16T12:59:46.100 INFO:teuthology.orchestra.run.smithi042.stdout:1847 write 0xe2ddbcc thru 0xe2eb811 (0xdc46 bytes) 2024-09-16T12:59:46.102 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:46.099+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:46.106 INFO:teuthology.orchestra.run.smithi042.stdout:1848 read 0xdf55abb thru 0xdf5a6f8 (0x4c3e bytes) 2024-09-16T12:59:46.106 INFO:teuthology.orchestra.run.smithi042.stdout:1849 write 0x49c4170 thru 0x49d3dce (0xfc5f bytes) 2024-09-16T12:59:46.112 INFO:teuthology.orchestra.run.smithi042.stdout:1850 write 0x4ab35c4 thru 0x4abb47d (0x7eba bytes) 2024-09-16T12:59:46.115 INFO:teuthology.orchestra.run.smithi042.stdout:1852 write 0xa57a452 thru 0xa57fa42 (0x55f1 bytes) 2024-09-16T12:59:46.121 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:46.121 INFO:teuthology.orchestra.run.smithi042.stdout:1855 read 0x8f84a3b thru 0x8f87524 (0x2aea bytes) 2024-09-16T12:59:46.121 INFO:teuthology.orchestra.run.smithi042.stdout:1856 punch from 0xcf83e03 to 0xcf90d99, (0xcf96 bytes) 2024-09-16T12:59:46.121 INFO:teuthology.orchestra.run.smithi042.stdout:1857 read 0xa8b3ad thru 0xa8c648 (0x129c bytes) 2024-09-16T12:59:46.122 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:46.122 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:46.122 INFO:teuthology.orchestra.run.smithi042.stdout:1860 read 0xc4c690e thru 0xc4d5f33 (0xf626 bytes) 2024-09-16T12:59:46.123 INFO:teuthology.orchestra.run.smithi042.stdout:1861 write 0x15378a8 thru 0x153b77b (0x3ed4 bytes) 2024-09-16T12:59:46.125 INFO:teuthology.orchestra.run.smithi042.stdout:1862 write 0x8aa6790 thru 0x8aa77ec (0x105d bytes) 2024-09-16T12:59:46.131 INFO:teuthology.orchestra.run.smithi042.stdout:1864 write 0x8b7d313 thru 0x8b87a34 (0xa722 bytes) 2024-09-16T12:59:46.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:46.133 INFO:teuthology.orchestra.run.smithi042.stdout:1866 writesame 0x2618944 thru 0x261f80f data_size 0xe(0x6ecc bytes) 2024-09-16T12:59:46.136 INFO:teuthology.orchestra.run.smithi042.stdout:1868 write 0x7e6f593 thru 0x7e7b553 (0xbfc1 bytes) 2024-09-16T12:59:46.140 INFO:teuthology.orchestra.run.smithi042.stdout:1869 writesame 0x2d4234c thru 0x2d4d9b2 data_size 0x37(0xb667 bytes) 2024-09-16T12:59:46.143 INFO:teuthology.orchestra.run.smithi042.stdout:1870 writesame 0xc59a81f thru 0xc5a2262 data_size 0x19(0x7a44 bytes) 2024-09-16T12:59:46.145 INFO:teuthology.orchestra.run.smithi042.stdout:1871 clone 15 order 20 su 262144 sc 6 2024-09-16T12:59:46.203 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:46.199+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:46.225 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:46.225 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:46.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:46.304 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:46.295+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:346 4.2 4:4550e1e5:::rbd_id.image_client.0-clone1:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.47800848, 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003768212, 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 346 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850712+0000", 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850713+0000", 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6270000000000001e-06 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850722+0000", 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2169999999999994e-06 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850739+0000", 2024-09-16T12:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7005000000000001e-05 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850758+0000", 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9502999999999999e-05 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850826+0000", 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8032000000000002e-05 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.850959+0000", 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013249600000000001 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.851164+0000", 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020556 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.852079+0000", 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091475699999999998 2024-09-16T12:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854331+0000", 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022523040000000001 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854400+0000", 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8356999999999996e-05 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:16.854480+0000", 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0353999999999997e-05 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:543 4.7 4:e655f1b3:::rbd_data.11b59d02cc38.000000000000004c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 367827~25389] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.093402067, 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004583953, 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 543 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235318+0000", 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235321+0000", 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.117e-06 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235354+0000", 2024-09-16T12:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2962000000000001e-05 2024-09-16T12:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235394+0000", 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9954999999999999e-05 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235409+0000", 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4756999999999999e-05 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235553+0000", 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000144284 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.235809+0000", 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025601299999999998 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.236081+0000", 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000272093 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.237888+0000", 2024-09-16T12:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806298 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239756+0000", 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001867884 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239842+0000", 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6581000000000003e-05 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.239902+0000", 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009000000000001e-05 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:556 4.7 4:e23e5852:::rbd_data.11b59d02cc38.0000000000000074:head [set-alloc-hint object_size 1048576 write_size 1048576,write 736209~50223] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.064059515, 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038485979999999999, 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 556 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264661+0000", 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264664+0000", 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264757+0000", 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2904000000000004e-05 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264792+0000", 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4437000000000002e-05 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264804+0000", 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.218e-05 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.264922+0000", 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118559 2024-09-16T12:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265276+0000", 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035357999999999999 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.265652+0000", 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037578600000000002 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.267338+0000", 2024-09-16T12:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016864899999999999 2024-09-16T12:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268352+0000", 2024-09-16T12:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00101337 2024-09-16T12:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268425+0000", 2024-09-16T12:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3398000000000004e-05 2024-09-16T12:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.268509+0000", 2024-09-16T12:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4318000000000001e-05 2024-09-16T12:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:574 4.7 4:e23e5852:::rbd_data.11b59d02cc38.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.016503747000002, 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051945439999999997, 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 574 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312217+0000", 2024-09-16T12:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3300000000000003e-07 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312221+0000", 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9550000000000002e-06 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312229+0000", 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6159999999999996e-06 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312233+0000", 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8779999999999997e-06 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312886+0000", 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065238099999999995 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313137+0000", 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025079300000000002 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313541+0000", 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00040414999999999998 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314738+0000", 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011973839999999999 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317292+0000", 2024-09-16T12:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002554312 2024-09-16T12:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317342+0000", 2024-09-16T12:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9175999999999998e-05 2024-09-16T12:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317411+0000", 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9466000000000002e-05 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:711 4.4 4:20cc0444:::rbd_data.11b59d02cc38.000000000000008f:head [set-alloc-hint object_size 1048576 write_size 1048576,write 655360~35331] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.640312231999999, 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038704080000000001, 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 711 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688408+0000", 2024-09-16T12:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688410+0000", 2024-09-16T12:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-16T12:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688470+0000", 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0034000000000002e-05 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688491+0000", 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605999999999999e-05 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688508+0000", 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7452000000000002e-05 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688639+0000", 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130541 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.688984+0000", 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034521699999999998 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.689500+0000", 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000516023 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692011+0000", 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025114349999999998 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692150+0000", 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013872999999999999 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692223+0000", 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2917999999999995e-05 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.692278+0000", 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5402999999999998e-05 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:715 4.4 4:2570fb00:::rbd_data.11b59d02cc38.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.627421837, 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037942140000000002, 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 715 2024-09-16T12:59:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701298+0000", 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701300+0000", 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9920000000000002e-06 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701311+0000", 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0361e-05 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701343+0000", 2024-09-16T12:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2181000000000002e-05 2024-09-16T12:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701361+0000", 2024-09-16T12:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8366999999999999e-05 2024-09-16T12:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701465+0000", 2024-09-16T12:59:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103854 2024-09-16T12:59:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.701709+0000", 2024-09-16T12:59:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024390300000000001 2024-09-16T12:59:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.702064+0000", 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035441500000000001 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.704221+0000", 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021577609999999998 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.704960+0000", 2024-09-16T12:59:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073849600000000005 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.705039+0000", 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9092999999999996e-05 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.705093+0000", 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3791e-05 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:826 4.4 4:22c6e589:::rbd_data.11b59d02cc38.000000000000005a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 196608~7156] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.399570023999999, 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043112890000000003, 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 826 2024-09-16T12:59:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929150+0000", 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929153+0000", 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.976e-06 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929175+0000", 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1897999999999999e-05 2024-09-16T12:59:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929213+0000", 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7528999999999999e-05 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929230+0000", 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7093999999999999e-05 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929357+0000", 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012754300000000001 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929610+0000", 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025261899999999999 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929979+0000", 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036918900000000002 2024-09-16T12:59:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.932855+0000", 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028755880000000001 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933292+0000", 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043741500000000003 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933360+0000", 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7533999999999994e-05 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933462+0000", 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010190400000000001 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:827 4.4 4:2570fb00:::rbd_data.11b59d02cc38.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write 155625~40983] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.399068866, 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045348059999999997, 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 827 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929651+0000", 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929654+0000", 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-06 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929755+0000", 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100955 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929783+0000", 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7902000000000002e-05 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.929798+0000", 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5381000000000001e-05 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930965+0000", 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011667310000000001 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.931262+0000", 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029734500000000002 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.931625+0000", 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036316799999999998 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933825+0000", 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021993970000000001 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934061+0000", 2024-09-16T12:59:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000236095 2024-09-16T12:59:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934138+0000", 2024-09-16T12:59:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7188999999999996e-05 2024-09-16T12:59:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.934186+0000", 2024-09-16T12:59:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8303000000000002e-05 2024-09-16T12:59:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:948 4.f 4:f1e0f236:::rbd_data.11b59d02cc38.000000000000008e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.140159604000001, 2024-09-16T12:59:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013393758, 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 948 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188561+0000", 2024-09-16T12:59:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188563+0000", 2024-09-16T12:59:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-16T12:59:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188576+0000", 2024-09-16T12:59:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3071000000000001e-05 2024-09-16T12:59:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188611+0000", 2024-09-16T12:59:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4623999999999999e-05 2024-09-16T12:59:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188624+0000", 2024-09-16T12:59:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2958e-05 2024-09-16T12:59:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188732+0000", 2024-09-16T12:59:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107911 2024-09-16T12:59:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.188873+0000", 2024-09-16T12:59:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014080700000000001 2024-09-16T12:59:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.192672+0000", 2024-09-16T12:59:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037992270000000001 2024-09-16T12:59:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201725+0000", 2024-09-16T12:59:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090529549999999997 2024-09-16T12:59:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201811+0000", 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6328000000000004e-05 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201894+0000", 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2991000000000006e-05 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.201954+0000", 2024-09-16T12:59:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0336e-05 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:999 4.f 4:f13e1533:::rbd_data.11b59d02cc38.0000000000000035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.019849341, 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041771059999999999, 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 999 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308871+0000", 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308872+0000", 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.514e-06 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308880+0000", 2024-09-16T12:59:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3300000000000001e-06 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308903+0000", 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3017999999999999e-05 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308915+0000", 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2636000000000001e-05 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309000+0000", 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4263000000000003e-05 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309082+0000", 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2443999999999999e-05 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309190+0000", 2024-09-16T12:59:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010767700000000001 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.310638+0000", 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001448509 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.312936+0000", 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002297669 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.312990+0000", 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3494e-05 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.313048+0000", 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8551999999999998e-05 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1164 4.2 4:400c9634:::rbd_header.11b53140b009:head [watch watch cookie 140717154602240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.564865402999999, 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0048381040000000002, 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1164 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763855+0000", 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763856+0000", 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.096e-06 2024-09-16T12:59:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763861+0000", 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3199999999999999e-06 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763877+0000", 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.537e-05 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763885+0000", 2024-09-16T12:59:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.687e-06 2024-09-16T12:59:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.763943+0000", 2024-09-16T12:59:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7243000000000001e-05 2024-09-16T12:59:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764001+0000", 2024-09-16T12:59:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8214000000000002e-05 2024-09-16T12:59:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764123+0000", 2024-09-16T12:59:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012186899999999999 2024-09-16T12:59:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.764972+0000", 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084946000000000002 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768511+0000", 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035386139999999998 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768579+0000", 2024-09-16T12:59:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8584999999999995e-05 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.768693+0000", 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113646 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1269 4.7 4:e145092b:::rbd_data.11b53140b009.00000000000000d3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 155648] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.279691752000002, 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035041009999999999, 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1269 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049029+0000", 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2699999999999999e-07 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049033+0000", 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629999999999997e-06 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049041+0000", 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9920000000000007e-06 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049056+0000", 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4344e-05 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049131+0000", 2024-09-16T12:59:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-16T12:59:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049343+0000", 2024-09-16T12:59:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021202300000000001 2024-09-16T12:59:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.049705+0000", 2024-09-16T12:59:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000362529 2024-09-16T12:59:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.050871+0000", 2024-09-16T12:59:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001165391 2024-09-16T12:59:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052300+0000", 2024-09-16T12:59:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014292459999999999 2024-09-16T12:59:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052430+0000", 2024-09-16T12:59:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013032399999999999 2024-09-16T12:59:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.052533+0000", 2024-09-16T12:59:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102215 2024-09-16T12:59:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1287 4.7 4:e145092b:::rbd_data.11b53140b009.00000000000000d3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.263850365, 2024-09-16T12:59:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042846999999999998, 2024-09-16T12:59:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1287 2024-09-16T12:59:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064870+0000", 2024-09-16T12:59:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064871+0000", 2024-09-16T12:59:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-07 2024-09-16T12:59:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064875+0000", 2024-09-16T12:59:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8029999999999996e-06 2024-09-16T12:59:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064889+0000", 2024-09-16T12:59:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3665e-05 2024-09-16T12:59:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064906+0000", 2024-09-16T12:59:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7343e-05 2024-09-16T12:59:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.064971+0000", 2024-09-16T12:59:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5147999999999994e-05 2024-09-16T12:59:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.065076+0000", 2024-09-16T12:59:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105004 2024-09-16T12:59:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.065203+0000", 2024-09-16T12:59:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126451 2024-09-16T12:59:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.066904+0000", 2024-09-16T12:59:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017010580000000001 2024-09-16T12:59:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069097+0000", 2024-09-16T12:59:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021933299999999998 2024-09-16T12:59:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069130+0000", 2024-09-16T12:59:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3161999999999999e-05 2024-09-16T12:59:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.069155+0000", 2024-09-16T12:59:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4161000000000001e-05 2024-09-16T12:59:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1306 4.12 4:4f3fb25c:::rbd_data.11b53140b009.00000000000000c9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 147456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.241720471000001, 2024-09-16T12:59:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038175800000000001, 2024-09-16T12:59:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1306 2024-09-16T12:59:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087000+0000", 2024-09-16T12:59:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087001+0000", 2024-09-16T12:59:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7699999999999992e-07 2024-09-16T12:59:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087006+0000", 2024-09-16T12:59:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5010000000000004e-06 2024-09-16T12:59:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087023+0000", 2024-09-16T12:59:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.662e-05 2024-09-16T12:59:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087029+0000", 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4860000000000002e-06 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087105+0000", 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5265000000000005e-05 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087480+0000", 2024-09-16T12:59:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037499899999999999 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087905+0000", 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042498600000000003 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.089239+0000", 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013342930000000001 2024-09-16T12:59:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090707+0000", 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001468235 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090771+0000", 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3553000000000006e-05 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090817+0000", 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6665000000000001e-05 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1307 4.7 4:e4cf6733:::rbd_data.11b53140b009.00000000000000ca:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 147456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.241664444000001, 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003717405, 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1307 2024-09-16T12:59:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087056+0000", 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087061+0000", 2024-09-16T12:59:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8919999999999999e-06 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087069+0000", 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1470000000000001e-06 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087074+0000", 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2400000000000001e-06 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087159+0000", 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5765000000000003e-05 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087484+0000", 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032450900000000001 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.087913+0000", 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042913400000000002 2024-09-16T12:59:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.089457+0000", 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001544028 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090619+0000", 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011616090000000001 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090710+0000", 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1416999999999996e-05 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:19.090773+0000", 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3089999999999994e-05 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(4.7 58/57 [PushOp(4:e6558db2:::rbd_data.11b517fb51dd.0000000000000002:9, version: 44'18, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:e6558db2:::rbd_data.11b517fb51dd.0000000000000002:9@44'18, size: 10485760, copy_subset: [0~10485760], clone_subset: {}, snapset: 9=[]:{9=[9]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T12:59:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:41.066487+0000", 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.2622334410000002, 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003994793, 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.066487+0000", 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.066487+0000", 2024-09-16T12:59:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.066488+0000", 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2100000000000001e-06 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.068341+0000", 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018531719999999999 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.068362+0000", 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0440000000000001e-05 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.068374+0000", 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2510000000000001e-05 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.068463+0000", 2024-09-16T12:59:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8677999999999994e-05 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.068473+0000", 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0304e-05 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.070482+0000", 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020084790000000001 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(4.14 58/57 [PushOp(4:2a646c7f:::rbd_data.11b5c378985b.0000000000000001:head, version: 41'9, data_included: [8192~8192,32768~16384,147456~16384,180224~8192,262144~16384,294912~8192,319488~8192,409600~8192,434176~16384,524288~8192,548864~16384,581632~8192,663552~9752,2097152~16384,2129920~8192,2154496~16384,2187264~8192,2211840~16384,2244608~8192,2269184~16384,2301952~8192,2326528~16384,2359296~8192,2383872~16384,2416640~8192,2441216~16384,2473984~8192,2498560~16384,2531328~8192,2555904~16384,2588672~8192,2613248~16384,2646016~8192,2670592~16384,2703360~8192,2727936~16384,2760704~8192,2785280~16384,2818048~8192,2842624~16384,2875392~8192,2899968~16384,2932736~8192,2957312~16384,2990080~8192], data_size: 542232, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a646c7f:::rbd_data.11b5c378985b.0000000000000001:head@41'9, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:41.103191+0000", 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.225529399, 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067820279999999998, 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.103191+0000", 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.103191+0000", 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.103192+0000", 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5000000000000001e-07 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.103801+0000", 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060953799999999996 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.103840+0000", 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9066999999999997e-05 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.103871+0000", 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0858000000000002e-05 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.103998+0000", 2024-09-16T12:59:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127143 2024-09-16T12:59:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.104004+0000", 2024-09-16T12:59:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6829999999999996e-06 2024-09-16T12:59:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:41.109973+0000", 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0059688889999999998 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:13800 4.f 4:f093f7c6:::rbd_data.11b51cc7997f.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:45.649035+0000", 2024-09-16T12:59:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.67968519199999999, 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063717679999999999, 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 13800 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.649035+0000", 2024-09-16T12:59:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.649035+0000", 2024-09-16T12:59:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.649038+0000", 2024-09-16T12:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-16T12:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.649054+0000", 2024-09-16T12:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.59e-05 2024-09-16T12:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.649093+0000", 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9236999999999997e-05 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.649106+0000", 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3404000000000001e-05 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.649182+0000", 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5885999999999998e-05 2024-09-16T12:59:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.649350+0000", 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016766 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.650629+0000", 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012787269999999999 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.655250+0000", 2024-09-16T12:59:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046208910000000002 2024-09-16T12:59:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.655295+0000", 2024-09-16T12:59:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5191999999999997e-05 2024-09-16T12:59:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.655359+0000", 2024-09-16T12:59:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4533e-05 2024-09-16T12:59:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.655407+0000", 2024-09-16T12:59:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7540999999999998e-05 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:13907 4.7 4:e39305e1:::rbd_data.11b51cc7997f.000000000000000b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:45.905640+0000", 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.42308039600000003, 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040289790000000002, 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 13907 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.905640+0000", 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.905640+0000", 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.905642+0000", 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6700000000000001e-06 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.905650+0000", 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1769999999999998e-06 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.905668+0000", 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8050000000000002e-05 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.905679+0000", 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1326e-05 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.905761+0000", 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2018e-05 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.905904+0000", 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014318100000000001 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.906187+0000", 2024-09-16T12:59:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028272900000000001 2024-09-16T12:59:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.907356+0000", 2024-09-16T12:59:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011689809999999999 2024-09-16T12:59:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.909526+0000", 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002169893 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.909595+0000", 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9443000000000006e-05 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.909669+0000", 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3510999999999995e-05 2024-09-16T12:59:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:13923 4.7 4:e6111db4:::rbd_data.11b51cc7997f.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:45.918157+0000", 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.41056310200000001, 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039436369999999998, 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 13923 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.918157+0000", 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.918157+0000", 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.918159+0000", 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.918309+0000", 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150106 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.918326+0000", 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7433999999999999e-05 2024-09-16T12:59:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.918335+0000", 2024-09-16T12:59:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3300000000000005e-06 2024-09-16T12:59:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.918451+0000", 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011528599999999999 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.918668+0000", 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021699600000000001 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.919087+0000", 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041907899999999999 2024-09-16T12:59:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.920863+0000", 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001776597 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.921959+0000", 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010952589999999999 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.922031+0000", 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2397999999999994e-05 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.922101+0000", 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9769000000000002e-05 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.480 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:46.482 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:46.482 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:46.479+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:46.583 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:46.575+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:46.594 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:46.594 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:46.594 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:46.594 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:46.604 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:46.684 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:46.679+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:46.709 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:46.710 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:46.710 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:46.710 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:46.710 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:46.719 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:46.784 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:46.778+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:46.824 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.037626848000002, 2024-09-16T12:59:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003284908, 2024-09-16T12:59:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 10 2024-09-16T12:59:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779657+0000", 2024-09-16T12:59:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1239999999999999e-06 2024-09-16T12:59:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779667+0000", 2024-09-16T12:59:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0487e-05 2024-09-16T12:59:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779683+0000", 2024-09-16T12:59:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6146000000000001e-05 2024-09-16T12:59:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779695+0000", 2024-09-16T12:59:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1347e-05 2024-09-16T12:59:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779769+0000", 2024-09-16T12:59:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.449e-05 2024-09-16T12:59:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779918+0000", 2024-09-16T12:59:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000148462 2024-09-16T12:59:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.780088+0000", 2024-09-16T12:59:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017066600000000001 2024-09-16T12:59:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.781019+0000", 2024-09-16T12:59:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093024399999999999 2024-09-16T12:59:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782840+0000", 2024-09-16T12:59:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018211379999999999 2024-09-16T12:59:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782899+0000", 2024-09-16T12:59:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8777000000000003e-05 2024-09-16T12:59:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782940+0000", 2024-09-16T12:59:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1026999999999999e-05 2024-09-16T12:59:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:502 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 726006~2122] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.673858915, 2024-09-16T12:59:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026083510000000001, 2024-09-16T12:59:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 502 2024-09-16T12:59:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143426+0000", 2024-09-16T12:59:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0460000000000001e-06 2024-09-16T12:59:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143465+0000", 2024-09-16T12:59:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T12:59:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143507+0000", 2024-09-16T12:59:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1678999999999999e-05 2024-09-16T12:59:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143523+0000", 2024-09-16T12:59:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6464e-05 2024-09-16T12:59:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143631+0000", 2024-09-16T12:59:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107357 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143882+0000", 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025138100000000001 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.144130+0000", 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024822700000000001 2024-09-16T12:59:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145527+0000", 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001396986 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145925+0000", 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000397964 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145985+0000", 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9896999999999999e-05 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.146031+0000", 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5938e-05 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:681 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [set-alloc-hint object_size 1048576 write_size 1048576,write 119408~11664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.208720778, 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035485299999999998, 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 681 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608564+0000", 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.146e-06 2024-09-16T12:59:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608592+0000", 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8585000000000002e-05 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608635+0000", 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629000000000001e-05 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608649+0000", 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4270000000000001e-05 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608797+0000", 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014800499999999999 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609162+0000", 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036455100000000001 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609411+0000", 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024895400000000002 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611008+0000", 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015968860000000001 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611948+0000", 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094050999999999998 2024-09-16T12:59:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612013+0000", 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5185999999999996e-05 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612109+0000", 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5808000000000002e-05 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:688 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.173391151000001, 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027962989999999999, 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 688 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643893+0000", 2024-09-16T12:59:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8200000000000001e-06 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643906+0000", 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643953+0000", 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6399000000000001e-05 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643972+0000", 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9130000000000001e-05 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644163+0000", 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019125899999999999 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644426+0000", 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026297899999999999 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644768+0000", 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000342574 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646240+0000", 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001471312 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646617+0000", 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037757399999999999 2024-09-16T12:59:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646658+0000", 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1083e-05 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646687+0000", 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8248000000000001e-05 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:810 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 443751~14451] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.920637247999998, 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036011279999999999, 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 810 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896648+0000", 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6109999999999998e-06 2024-09-16T12:59:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896701+0000", 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3168999999999999e-05 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896743+0000", 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1445e-05 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896758+0000", 2024-09-16T12:59:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5591e-05 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896878+0000", 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120256 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897013+0000", 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134327 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897221+0000", 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020804800000000001 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.899109+0000", 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018886899999999999 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900084+0000", 2024-09-16T12:59:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097495899999999996 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900178+0000", 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3264999999999996e-05 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900245+0000", 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7767000000000004e-05 2024-09-16T12:59:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:826 4.4 e38/31 4:22c6e589:::rbd_data.11b59d02cc38.000000000000005a:head v 38'8, mlcod=37'6)", 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.887118693000001, 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028983849999999999, 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930165+0000", 2024-09-16T12:59:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.114e-06 2024-09-16T12:59:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930191+0000", 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6057000000000001e-05 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930222+0000", 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0959e-05 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930235+0000", 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-16T12:59:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930349+0000", 2024-09-16T12:59:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011372200000000001 2024-09-16T12:59:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930402+0000", 2024-09-16T12:59:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2766000000000003e-05 2024-09-16T12:59:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.932954+0000", 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025517980000000001 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933061+0000", 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010770800000000001 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:829 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.875412579999999, 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024711590000000001, 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 829 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941869+0000", 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941872+0000", 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.225e-06 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941889+0000", 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6487999999999999e-05 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941926+0000", 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7727e-05 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.941941+0000", 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5085999999999999e-05 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942053+0000", 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011112600000000001 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942210+0000", 2024-09-16T12:59:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000157653 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.942387+0000", 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000176959 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.943529+0000", 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001141801 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944262+0000", 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073265199999999998 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944304+0000", 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2200999999999998e-05 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.944340+0000", 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6241000000000001e-05 2024-09-16T12:59:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:860 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [set-alloc-hint object_size 1048576 write_size 1048576,write 997016~9314] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.819006120000001, 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037930170000000001, 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 860 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998280+0000", 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0969999999999999e-06 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998307+0000", 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7804999999999999e-05 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998349+0000", 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1838000000000002e-05 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998364+0000", 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5194000000000001e-05 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998511+0000", 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014693900000000001 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998912+0000", 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000401066 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.999376+0000", 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000463938 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001244+0000", 2024-09-16T12:59:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018672389999999999 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001920+0000", 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067617100000000004 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002008+0000", 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8342000000000002e-05 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002068+0000", 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0387999999999998e-05 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:995 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510125027000001, 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040928859999999996, 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 995 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307159+0000", 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9739999999999998e-06 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307174+0000", 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4134000000000001e-05 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307211+0000", 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7385000000000002e-05 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307230+0000", 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8961e-05 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307461+0000", 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230721 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307710+0000", 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024951200000000002 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308016+0000", 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030602199999999998 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309601+0000", 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015845200000000001 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311094+0000", 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00149321 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311180+0000", 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6142000000000003e-05 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311249+0000", 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9305000000000001e-05 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3720 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.922156381000001, 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024764850000000001, 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3720 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895125+0000", 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895128+0000", 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895141+0000", 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2824e-05 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895170+0000", 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8992e-05 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895181+0000", 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1106e-05 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895274+0000", 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3073999999999997e-05 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895417+0000", 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014300599999999999 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.895810+0000", 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039280400000000002 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897118+0000", 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00130792 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897512+0000", 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039389699999999999 2024-09-16T12:59:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897562+0000", 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0229000000000003e-05 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.897602+0000", 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9892999999999999e-05 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3724 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.906486215000001, 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002592624, 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910798+0000", 2024-09-16T12:59:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7350000000000001e-06 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910813+0000", 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4469e-05 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910847+0000", 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3986999999999998e-05 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910869+0000", 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2764000000000001e-05 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910939+0000", 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0086000000000006e-05 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.911119+0000", 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179598 2024-09-16T12:59:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.911351+0000", 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023168200000000001 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.912688+0000", 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013376729999999999 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913294+0000", 2024-09-16T12:59:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060521299999999995 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913343+0000", 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9537999999999999e-05 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913388+0000", 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4879000000000002e-05 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4040 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.354956494, 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026268979999999999, 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4040 2024-09-16T12:59:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462327+0000", 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0229999999999999e-06 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462340+0000", 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3189e-05 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462373+0000", 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2555000000000003e-05 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462386+0000", 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2935000000000001e-05 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462458+0000", 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1847999999999994e-05 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462582+0000", 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012399199999999999 2024-09-16T12:59:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.463055+0000", 2024-09-16T12:59:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047307700000000002 2024-09-16T12:59:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464541+0000", 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014867649999999999 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464841+0000", 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029975500000000002 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464912+0000", 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0344000000000003e-05 2024-09-16T12:59:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464952+0000", 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0414999999999998e-05 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4042 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.348595191000001, 2024-09-16T12:59:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002527208, 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4042 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468688+0000", 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9549999999999999e-06 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468700+0000", 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1443e-05 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468726+0000", 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6591999999999999e-05 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468737+0000", 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0951999999999999e-05 2024-09-16T12:59:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468841+0000", 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010335 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468995+0000", 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015392800000000001 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.469556+0000", 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00056112700000000002 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471063+0000", 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001507226 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471130+0000", 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6693999999999995e-05 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471176+0000", 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6223000000000001e-05 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471214+0000", 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7718e-05 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4044 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.342827478, 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002965217, 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4044 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474456+0000", 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.818e-06 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474465+0000", 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5459999999999997e-06 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474490+0000", 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5058e-05 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474502+0000", 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1110999999999999e-05 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474610+0000", 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108606 2024-09-16T12:59:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474815+0000", 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020453800000000001 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.475681+0000", 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086620700000000002 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477079+0000", 2024-09-16T12:59:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001398388 2024-09-16T12:59:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477340+0000", 2024-09-16T12:59:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026078800000000002 2024-09-16T12:59:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477385+0000", 2024-09-16T12:59:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5014e-05 2024-09-16T12:59:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477419+0000", 2024-09-16T12:59:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4143000000000002e-05 2024-09-16T12:59:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4069 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.281452573999999, 2024-09-16T12:59:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025697279999999999, 2024-09-16T12:59:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4069 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535831+0000", 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8109999999999999e-06 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535841+0000", 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0267e-05 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535862+0000", 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0693e-05 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535869+0000", 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7309999999999993e-06 2024-09-16T12:59:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535958+0000", 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8721000000000007e-05 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.536094+0000", 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013532999999999999 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.536561+0000", 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000467057 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538128+0000", 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001567337 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538285+0000", 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015667900000000001 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538355+0000", 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9991000000000001e-05 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538399+0000", 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4110999999999998e-05 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:5580 4.12 e46/40 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head v 46'11, mlcod=46'9)", 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.108287346000001, 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039131280000000001, 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708997+0000", 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4109999999999999e-06 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710732+0000", 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001735002 2024-09-16T12:59:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710766+0000", 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-05 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710779+0000", 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3137e-05 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710905+0000", 2024-09-16T12:59:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012594799999999999 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710972+0000", 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7062000000000003e-05 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.712815+0000", 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018428419999999999 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.712907+0000", 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2516000000000007e-05 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6800 4.1c 4:3d97daaf:::rbd_data.11b58f77e1c4.0000000000000030:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1985195~61789] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:32.580187+0000", 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.237094337, 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002873221, 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6800 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580187+0000", 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580187+0000", 2024-09-16T12:59:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580190+0000", 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8880000000000001e-06 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580275+0000", 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4672000000000005e-05 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580313+0000", 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7991000000000003e-05 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580330+0000", 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7425e-05 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580416+0000", 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5928999999999997e-05 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580627+0000", 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021073899999999999 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580877+0000", 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024978100000000003 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.582227+0000", 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013502729999999999 2024-09-16T12:59:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.582955+0000", 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072785299999999999 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.583015+0000", 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0149999999999998e-05 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.583060+0000", 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5519999999999998e-05 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6806 4.1c 4:3d97daaf:::rbd_data.11b58f77e1c4.0000000000000030:head [stat,truncate 2002817] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:32.608984+0000", 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.208297118000001, 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027044930000000001, 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6806 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.608984+0000", 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.608984+0000", 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.608989+0000", 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2359999999999998e-06 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609009+0000", 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0757000000000001e-05 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609069+0000", 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0019000000000003e-05 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609094+0000", 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4119999999999999e-05 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609229+0000", 2024-09-16T12:59:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000135773 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609396+0000", 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000166501 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609647+0000", 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025091499999999999 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611408+0000", 2024-09-16T12:59:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017616019999999999 2024-09-16T12:59:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611590+0000", 2024-09-16T12:59:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181192 2024-09-16T12:59:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611643+0000", 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3368000000000001e-05 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611689+0000", 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6010000000000002e-05 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7104 4.1c 4:394c1983:::rbd_data.11b58f77e1c4.0000000000000024:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.770772183, 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031870290000000001, 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7104 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046511+0000", 2024-09-16T12:59:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9819999999999998e-06 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046518+0000", 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4489999999999996e-06 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046536+0000", 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8544999999999999e-05 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046544+0000", 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.729e-06 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046624+0000", 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9708000000000003e-05 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046755+0000", 2024-09-16T12:59:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130846 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049696+0000", 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029417699999999998 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:8852 4.d e55/31 4:b280c692:::rbd_data.11b55eb65e78.0000000000000004:head v 55'43, mlcod=54'40)", 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:38.090571+0000", 2024-09-16T12:59:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.7267100630000005, 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002909961, 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090571+0000", 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090571+0000", 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090573+0000", 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.274e-06 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090588+0000", 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5109999999999999e-05 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090613+0000", 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4916999999999999e-05 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090625+0000", 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2354000000000001e-05 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090760+0000", 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134468 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090826+0000", 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6765000000000002e-05 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.093373+0000", 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025467110000000001 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.093481+0000", 2024-09-16T12:59:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108362 2024-09-16T12:59:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:46.958 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:46.959 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:46.960 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:46.954+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:47.034 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T12:59:47.060 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:47.054+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:47.077 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:47.077 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:47.077 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:47.077 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:47.087 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:47.161 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:47.151+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:47.168 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone14 intact 2024-09-16T12:59:47.184 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:47.184 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:47.184 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:47.185 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:47.185 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:47.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:47.262 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:47.255+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:47.330 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:47.357 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:47.357 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:47.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:47.357 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:47.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.357 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:575 4.17 4:eab06920:::rbd_data.11b59d02cc38.0000000000000075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:47.357 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:47.357 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.038464490999999, 2024-09-16T12:59:47.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049089249999999997, 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 575 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312344+0000", 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312345+0000", 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.353e-06 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312352+0000", 2024-09-16T12:59:47.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7499999999999997e-06 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312370+0000", 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8210999999999999e-05 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312391+0000", 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0968e-05 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.312965+0000", 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057344399999999999 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313134+0000", 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016979199999999999 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.313322+0000", 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018742300000000001 2024-09-16T12:59:47.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.314160+0000", 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083821799999999995 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317190+0000", 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030301339999999999 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317219+0000", 2024-09-16T12:59:47.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8368999999999999e-05 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.317253+0000", 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4263e-05 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:950 4.17 4:e9df2e13:::rbd_data.11b59d02cc38.0000000000000007:head [write 983040~36251] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.202396+0000", 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.148412779000001, 2024-09-16T12:59:47.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043336290000000003, 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 950 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202396+0000", 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202396+0000", 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202397+0000", 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7069999999999999e-06 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202433+0000", 2024-09-16T12:59:47.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5688000000000001e-05 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202452+0000", 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8550999999999999e-05 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202472+0000", 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0565999999999999e-05 2024-09-16T12:59:47.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202586+0000", 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113328 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.202802+0000", 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021687200000000001 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.203171+0000", 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036829700000000001 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.204781+0000", 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016104769999999999 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.206481+0000", 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017001989999999999 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.206620+0000", 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139004 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.206729+0000", 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010894 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:985 4.11 4:882565b9:::rbd_data.11b59d02cc38.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:47.369 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.304439+0000", 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.046369722000001, 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042693189999999997, 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 985 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304439+0000", 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304439+0000", 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304443+0000", 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-06 2024-09-16T12:59:47.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304464+0000", 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0767999999999998e-05 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304526+0000", 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1865999999999993e-05 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304575+0000", 2024-09-16T12:59:47.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9178999999999998e-05 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304715+0000", 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014015699999999999 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.304920+0000", 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020457299999999999 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.305215+0000", 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029488399999999998 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307142+0000", 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019277599999999999 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308529+0000", 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001386424 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308632+0000", 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010343700000000001 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308708+0000", 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5871000000000006e-05 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1001 4.1d 4:b8dd3499:::rbd_data.11b59d02cc38.0000000000000051:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.309177+0000", 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.041631528, 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049686330000000001, 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1001 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309177+0000", 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309177+0000", 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309179+0000", 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1320000000000001e-06 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309187+0000", 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2479999999999996e-06 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309216+0000", 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8884e-05 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309228+0000", 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1992e-05 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309365+0000", 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013697399999999999 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309849+0000", 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048348 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309985+0000", 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013675300000000001 2024-09-16T12:59:47.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311601+0000", 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001615472 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.314014+0000", 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024127950000000001 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.314102+0000", 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8324000000000003e-05 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.314146+0000", 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3578999999999997e-05 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1145 4.17 4:edd7f3ed:::image_client.0-clone2.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.733224+0000", 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.617584121, 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053216499999999998, 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1145 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733224+0000", 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733224+0000", 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733226+0000", 2024-09-16T12:59:47.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0109999999999999e-06 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733234+0000", 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3440000000000003e-06 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733255+0000", 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1347000000000001e-05 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733266+0000", 2024-09-16T12:59:47.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.117e-05 2024-09-16T12:59:47.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733330+0000", 2024-09-16T12:59:47.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3577000000000005e-05 2024-09-16T12:59:47.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:47.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.733339+0000", 2024-09-16T12:59:47.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0520000000000007e-06 2024-09-16T12:59:47.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.738402+0000", 2024-09-16T12:59:47.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050630680000000004 2024-09-16T12:59:47.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.738546+0000", 2024-09-16T12:59:47.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014408100000000001 2024-09-16T12:59:47.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.388 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.389 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:1148 4.0 4:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T12:59:47.389 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.741509+0000", 2024-09-16T12:59:47.389 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.6092996, 2024-09-16T12:59:47.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055772950000000003, 2024-09-16T12:59:47.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.389 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.390 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.390 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.390 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1148 2024-09-16T12:59:47.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.390 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741509+0000", 2024-09-16T12:59:47.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741509+0000", 2024-09-16T12:59:47.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741511+0000", 2024-09-16T12:59:47.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8029999999999999e-06 2024-09-16T12:59:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741519+0000", 2024-09-16T12:59:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0779999999999996e-06 2024-09-16T12:59:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741541+0000", 2024-09-16T12:59:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2609999999999999e-05 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741552+0000", 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1001e-05 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741622+0000", 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.004e-05 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741786+0000", 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000163251 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.741966+0000", 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018033400000000001 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.742883+0000", 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091699900000000005 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.746880+0000", 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039970769999999999 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.747012+0000", 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000131698 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.747086+0000", 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4404000000000001e-05 2024-09-16T12:59:47.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:3502 4.1b e41/31 4:dbfa2cf4:::rbd_data.11b5c378985b.0000000000000013:head v 41'8, mlcod=38'6)", 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:21.969284+0000", 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.381524611, 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060809890000000002, 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.969284+0000", 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.969284+0000", 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.969286+0000", 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2079999999999999e-06 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971841+0000", 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025551620000000001 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971871+0000", 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0230999999999999e-05 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971883+0000", 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1046000000000001e-05 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.971961+0000", 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8560999999999995e-05 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.972001+0000", 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0204999999999999e-05 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.975320+0000", 2024-09-16T12:59:47.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033191800000000001 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:21.975365+0000", 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4396e-05 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3536 4.14 4:2a646c7f:::rbd_data.11b5c378985b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.025341+0000", 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.325467396000001, 2024-09-16T12:59:47.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067260820000000004, 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3536 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025341+0000", 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025341+0000", 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025343+0000", 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.739e-06 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025571+0000", 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022837000000000001 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025590+0000", 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8360999999999999e-05 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025597+0000", 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1910000000000003e-06 2024-09-16T12:59:47.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025667+0000", 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0723999999999996e-05 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.025848+0000", 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000180679 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.027038+0000", 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001189854 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.031540+0000", 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045024829999999998 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.031922+0000", 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000381689 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.032005+0000", 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3141999999999998e-05 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.032067+0000", 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1849999999999999e-05 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.407 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6872 4.17 4:eebd2e7d:::rbd_data.11b58f77e1c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:32.718189+0000", 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.632619566000001, 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051813229999999998, 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6872 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718189+0000", 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718189+0000", 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718191+0000", 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9400000000000001e-06 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718200+0000", 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0890000000000006e-06 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718225+0000", 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4746999999999999e-05 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718235+0000", 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9760000000000007e-06 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718302+0000", 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7163999999999996e-05 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.718489+0000", 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000187677 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.719451+0000", 2024-09-16T12:59:47.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096120600000000004 2024-09-16T12:59:47.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.723228+0000", 2024-09-16T12:59:47.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003777772 2024-09-16T12:59:47.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.723267+0000", 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8374000000000002e-05 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.723328+0000", 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0778e-05 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.723370+0000", 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2599999999999999e-05 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.413 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7072 4.17 4:ee8c78e5:::rbd_object_map.11b58f77e1c4.000000000000000c:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.038229+0000", 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.312579094, 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053143549999999998, 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7072 2024-09-16T12:59:47.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038229+0000", 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038229+0000", 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038231+0000", 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7320000000000001e-06 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038239+0000", 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0199999999999994e-06 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038259+0000", 2024-09-16T12:59:47.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0228e-05 2024-09-16T12:59:47.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038268+0000", 2024-09-16T12:59:47.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8859999999999993e-06 2024-09-16T12:59:47.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038336+0000", 2024-09-16T12:59:47.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7878000000000003e-05 2024-09-16T12:59:47.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.038492+0000", 2024-09-16T12:59:47.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000156203 2024-09-16T12:59:47.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039076+0000", 2024-09-16T12:59:47.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058399299999999999 2024-09-16T12:59:47.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.041475+0000", 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023988569999999999 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043381+0000", 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019060119999999999 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043482+0000", 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100463 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043544+0000", 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2082999999999995e-05 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7075 4.19 4:9a86cce4:::rbd_data.11b58f77e1c4.000000000000001e:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:47.422 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.039147+0000", 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.311661814000001, 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045161819999999997, 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7075 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039147+0000", 2024-09-16T12:59:47.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039147+0000", 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039148+0000", 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0210000000000001e-06 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039153+0000", 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3449999999999996e-06 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039170+0000", 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7011e-05 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039177+0000", 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8229999999999997e-06 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039255+0000", 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8360000000000002e-05 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039578+0000", 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032231700000000002 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043663+0000", 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004085305 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7076 4.17 4:ee8c78e5:::rbd_object_map.11b58f77e1c4.000000000000000c:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.039494+0000", 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.311314764, 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055361380000000003, 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7076 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039494+0000", 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039494+0000", 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039495+0000", 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.569999999999999e-07 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039500+0000", 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5729999999999998e-06 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039513+0000", 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2505e-05 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039519+0000", 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9850000000000003e-06 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039814+0000", 2024-09-16T12:59:47.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000294803 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.039987+0000", 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017362200000000001 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040675+0000", 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00068771100000000003 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.043937+0000", 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032622720000000001 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044428+0000", 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049094700000000004 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044554+0000", 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012569400000000001 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045030+0000", 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047606900000000003 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7078 4.17 4:ee8c78e5:::rbd_object_map.11b58f77e1c4.000000000000000c:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.040912+0000", 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.309896834, 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044986549999999998, 2024-09-16T12:59:47.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7078 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040912+0000", 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040912+0000", 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040913+0000", 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1629999999999999e-06 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040919+0000", 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7610000000000004e-06 2024-09-16T12:59:47.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040932+0000", 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.040939+0000", 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5729999999999996e-06 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.041520+0000", 2024-09-16T12:59:47.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058121099999999999 2024-09-16T12:59:47.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.041687+0000", 2024-09-16T12:59:47.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000167694 2024-09-16T12:59:47.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.042457+0000", 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076913899999999996 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044193+0000", 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017368419999999999 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045063+0000", 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086991599999999998 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045349+0000", 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028614200000000003 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045410+0000", 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0800000000000001e-05 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.439 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7095 4.17 4:ef19ffb6:::rbd_data.11b58f77e1c4.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.044709+0000", 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.306099465000001, 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005134502, 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7095 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044709+0000", 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044709+0000", 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044710+0000", 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2240000000000001e-06 2024-09-16T12:59:47.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044728+0000", 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7884e-05 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044753+0000", 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4905e-05 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044764+0000", 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0841e-05 2024-09-16T12:59:47.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045485+0000", 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072079700000000002 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.045650+0000", 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016573 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046549+0000", 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000899024 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049015+0000", 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024659930000000001 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049745+0000", 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072982600000000004 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049806+0000", 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1106999999999996e-05 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049844+0000", 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7171e-05 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.445 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7096 4.17 4:ef19ffb6:::rbd_data.11b58f77e1c4.0000000000000018:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.044779+0000", 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.306029597, 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061860320000000002, 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7096 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044779+0000", 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044779+0000", 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044780+0000", 2024-09-16T12:59:47.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1100000000000005e-07 2024-09-16T12:59:47.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044784+0000", 2024-09-16T12:59:47.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.42e-06 2024-09-16T12:59:47.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044795+0000", 2024-09-16T12:59:47.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0431000000000001e-05 2024-09-16T12:59:47.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.044799+0000", 2024-09-16T12:59:47.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.797e-06 2024-09-16T12:59:47.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.047738+0000", 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002938689 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.047939+0000", 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020134600000000001 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.048152+0000", 2024-09-16T12:59:47.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021225000000000001 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.050856+0000", 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027047529999999998 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.050901+0000", 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4799000000000003e-05 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.050944+0000", 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2574000000000003e-05 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.050965+0000", 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1162000000000002e-05 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8149 4.f 4:f373df67:::rbd_header.11b59378797:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:47.452 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:35.940128+0000", 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.410680171999999, 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084867789999999999, 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8149 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940128+0000", 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940128+0000", 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940131+0000", 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4389999999999999e-06 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940141+0000", 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0540000000000001e-05 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940164+0000", 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2872000000000001e-05 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940176+0000", 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1462e-05 2024-09-16T12:59:47.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940245+0000", 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9686999999999999e-05 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.940253+0000", 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0250000000000002e-06 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.944461+0000", 2024-09-16T12:59:47.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042077379999999999 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.944583+0000", 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121668 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.944970+0000", 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038757599999999999 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.945593+0000", 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062260699999999996 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.948452+0000", 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002858817 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.948547+0000", 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5495000000000006e-05 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:35.948615+0000", 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7853000000000002e-05 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8579 4.11 4:895e0e12:::rbd_data.11b55eb65e78.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.742439+0000", 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.608369099000001, 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025774687000000001, 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8579 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742439+0000", 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742439+0000", 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742443+0000", 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.473e-06 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742461+0000", 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8329999999999999e-05 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742504+0000", 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3098e-05 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742520+0000", 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5982999999999999e-05 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742635+0000", 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000114537 2024-09-16T12:59:47.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742782+0000", 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014705100000000001 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.749390+0000", 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066079370000000004 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.767807+0000", 2024-09-16T12:59:47.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018417086999999999 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.767925+0000", 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011807300000000001 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.768098+0000", 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172876 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.768214+0000", 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116242 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8611 4.11 4:895e0e12:::rbd_data.11b55eb65e78.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.815402+0000", 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.535406570999999, 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927149, 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8611 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815402+0000", 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815402+0000", 2024-09-16T12:59:47.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815405+0000", 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8030000000000001e-06 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815416+0000", 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1741e-05 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815463+0000", 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6303e-05 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815474+0000", 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1189000000000001e-05 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815580+0000", 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010592899999999999 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815765+0000", 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185095 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.821526+0000", 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057609820000000004 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845015+0000", 2024-09-16T12:59:47.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023488841999999999 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845048+0000", 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2913999999999997e-05 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845204+0000", 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015631600000000001 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845329+0000", 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012503500000000001 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4533.0:8611 4.11 e54/31 ondisk, result = 0)", 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.839266+0000", 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.511542548, 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057569609999999997, 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839266+0000", 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839266+0000", 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839267+0000", 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7400000000000002e-07 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839275+0000", 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1699999999999997e-06 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839290+0000", 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5107e-05 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.839296+0000", 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4320000000000004e-06 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.844965+0000", 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056682069999999998 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.844990+0000", 2024-09-16T12:59:47.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.516e-05 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845023+0000", 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3011e-05 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:13818 4.11 4:8dc8e4d5:::rbd_data.11b51cc7997f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:45.688547+0000", 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.6622611379999999, 2024-09-16T12:59:47.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.032774062999999999, 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 13818 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.688547+0000", 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.688547+0000", 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.688550+0000", 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5799999999999999e-06 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.689948+0000", 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013981040000000001 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.689981+0000", 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3479999999999998e-05 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.689994+0000", 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2286000000000001e-05 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.690084+0000", 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9818e-05 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:47.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.690297+0000", 2024-09-16T12:59:47.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021293899999999999 2024-09-16T12:59:47.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:47.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.701762+0000", 2024-09-16T12:59:47.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011465145 2024-09-16T12:59:47.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:47.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721206+0000", 2024-09-16T12:59:47.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019444110000000001 2024-09-16T12:59:47.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:47.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721217+0000", 2024-09-16T12:59:47.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1452e-05 2024-09-16T12:59:47.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721263+0000", 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6097999999999998e-05 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721321+0000", 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8050999999999997e-05 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:47.481 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:47.482 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:47.483 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:47.483 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-16T12:59:47.484 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:47.484 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:47.478+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:47.494 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:47.508 INFO:teuthology.orchestra.run.smithi042.stdout:1873 read 0xafff66 thru 0xb08267 (0x8302 bytes) 2024-09-16T12:59:47.510 INFO:teuthology.orchestra.run.smithi042.stdout:1874 write 0x30fd91e thru 0x310c21c (0xe8ff bytes) 2024-09-16T12:59:47.513 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:47.536 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:47.539 INFO:teuthology.orchestra.run.smithi042.stdout:1875 trunc from 0xe2eb812 to 0x6bd2fb0 2024-09-16T12:59:47.558 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:47.571 INFO:teuthology.orchestra.run.smithi042.stdout:1876 write 0x8382913 thru 0x838970a (0x6df8 bytes) 2024-09-16T12:59:47.577 INFO:teuthology.orchestra.run.smithi042.stdout:1877 punch from 0x668380a to 0x6687238, (0x3a2e bytes) 2024-09-16T12:59:47.578 INFO:teuthology.orchestra.run.smithi042.stdout:1881 writesame 0x75eb816 thru 0x75f4d44 data_size 0x1(0x952f bytes) 2024-09-16T12:59:47.583 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:47.584 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:47.578+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:47.597 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:47.597 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:47.604 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:47.607 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:47.612 INFO:teuthology.orchestra.run.smithi042.stdout:1883 read 0x15ef407 thru 0x15f5015 (0x5c0f bytes) 2024-09-16T12:59:47.613 INFO:teuthology.orchestra.run.smithi042.stdout:1884 writesame 0x712b27e thru 0x712ca0c data_size 0x1(0x178f bytes) 2024-09-16T12:59:47.622 INFO:teuthology.orchestra.run.smithi042.stdout:1885 punch from 0x8757f5 to 0x8761a0, (0x9ab bytes) 2024-09-16T12:59:47.622 INFO:teuthology.orchestra.run.smithi042.stdout:1886 read 0x62fa2e6 thru 0x62ff0f3 (0x4e0e bytes) 2024-09-16T12:59:47.623 INFO:teuthology.orchestra.run.smithi042.stdout:1887 read 0x2c85cca thru 0x2c89e0c (0x4143 bytes) 2024-09-16T12:59:47.623 INFO:teuthology.orchestra.run.smithi042.stdout:1888 read 0x738e4cd thru 0x7397c85 (0x97b9 bytes) 2024-09-16T12:59:47.628 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:47.631 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:47.645 INFO:teuthology.orchestra.run.smithi042.stdout:1889 writesame 0xe79add3 thru 0xe7a1d35 data_size 0x5(0x6f63 bytes) 2024-09-16T12:59:47.645 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:47.646 INFO:teuthology.orchestra.run.smithi042.stdout:1893 punch from 0xe463662 to 0xe46dbc6, (0xa564 bytes) 2024-09-16T12:59:47.646 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:47.646 INFO:teuthology.orchestra.run.smithi042.stdout:1896 read 0x233e6dc thru 0x2348a98 (0xa3bd bytes) 2024-09-16T12:59:47.648 INFO:teuthology.orchestra.run.smithi042.stdout:1897 trunc from 0xe7a1d36 to 0xd724253 2024-09-16T12:59:47.659 INFO:teuthology.orchestra.run.smithi042.stdout:1899 punch from 0x56cbeee to 0x56ce0bb, (0x21cd bytes) 2024-09-16T12:59:47.659 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:47.660 INFO:teuthology.orchestra.run.smithi042.stdout:1901 trunc from 0xd724253 to 0xccb288d 2024-09-16T12:59:47.671 INFO:teuthology.orchestra.run.smithi042.stdout:1903 writesame 0x5aeeec7 thru 0x5af90f3 data_size 0x15(0xa22d bytes) 2024-09-16T12:59:47.680 INFO:teuthology.orchestra.run.smithi042.stdout:1906 read 0x3da8e60 thru 0x3dae17c (0x531d bytes) 2024-09-16T12:59:47.681 INFO:teuthology.orchestra.run.smithi042.stdout:1908 punch from 0x4dcf858 to 0x4dd1100, (0x18a8 bytes) 2024-09-16T12:59:47.681 INFO:teuthology.orchestra.run.smithi042.stdout:1909 writesame 0x77ac7a6 thru 0x77baab6 data_size 0x1(0xe311 bytes) 2024-09-16T12:59:47.685 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:47.678+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:47.699 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:47.699 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:47.709 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:47.721 INFO:teuthology.orchestra.run.smithi042.stdout:1910 punch from 0x6eda084 to 0x6ee0657, (0x65d3 bytes) 2024-09-16T12:59:47.725 INFO:teuthology.orchestra.run.smithi042.stdout:1911 trunc from 0xccb288d to 0xe4af99f 2024-09-16T12:59:47.736 INFO:teuthology.orchestra.run.smithi042.stdout:1912 read 0x1606335 thru 0x160beba (0x5b86 bytes) 2024-09-16T12:59:47.737 INFO:teuthology.orchestra.run.smithi042.stdout:1913 write 0x5f468d8 thru 0x5f4f427 (0x8b50 bytes) 2024-09-16T12:59:47.740 INFO:teuthology.orchestra.run.smithi042.stdout:1914 punch from 0x612fa8a to 0x6139f69, (0xa4df bytes) 2024-09-16T12:59:47.757 INFO:teuthology.orchestra.run.smithi042.stdout:1919 trunc from 0xe4af99f to 0x6a973d1 2024-09-16T12:59:47.785 INFO:teuthology.orchestra.run.smithi042.stdout:1920 read 0x3d742a7 thru 0x3d7cc4e (0x89a8 bytes) 2024-09-16T12:59:47.786 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:47.779+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:47.793 INFO:teuthology.orchestra.run.smithi042.stdout:1921 trunc from 0x6a973d1 to 0x32c82d6 2024-09-16T12:59:47.813 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:47.813 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:1922 read 0x18632a7 thru 0x1868378 (0x50d2 bytes) 2024-09-16T12:59:47.822 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:47.823 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:47.823 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:47.823 INFO:teuthology.orchestra.run.smithi042.stdout:1926 punch from 0xefb513 to 0xf00bc3, (0x56b0 bytes) 2024-09-16T12:59:47.823 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:47.844 INFO:teuthology.orchestra.run.smithi042.stdout:1929 write 0x7a926a1 thru 0x7aa23d9 (0xfd39 bytes) 2024-09-16T12:59:47.850 INFO:teuthology.orchestra.run.smithi042.stdout:1932 writesame 0x2f923f1 thru 0x2f9772c data_size 0xe(0x533c bytes) 2024-09-16T12:59:47.859 INFO:teuthology.orchestra.run.smithi042.stdout:1933 punch from 0x688c545 to 0x6899ab0, (0xd56b bytes) 2024-09-16T12:59:47.880 INFO:teuthology.orchestra.run.smithi042.stdout:1934 write 0xbf731f3 thru 0xbf80d19 (0xdb27 bytes) 2024-09-16T12:59:47.886 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:47.879+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:47.888 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:47.888 INFO:teuthology.orchestra.run.smithi042.stdout:1937 write 0x5581e35 thru 0x5584b85 (0x2d51 bytes) 2024-09-16T12:59:47.894 INFO:teuthology.orchestra.run.smithi042.stdout:1939 read 0x2df0317 thru 0x2dff3e6 (0xf0d0 bytes) 2024-09-16T12:59:47.895 INFO:teuthology.orchestra.run.smithi042.stdout:1940 writesame 0x98d2a6f thru 0x98d4c8b data_size 0x3(0x221d bytes) 2024-09-16T12:59:47.903 INFO:teuthology.orchestra.run.smithi042.stdout:1941 read 0x9bd8cb0 thru 0x9bde32f (0x5680 bytes) 2024-09-16T12:59:47.903 INFO:teuthology.orchestra.run.smithi042.stdout:1942 punch from 0x6f615ed to 0x6f6957a, (0x7f8d bytes) 2024-09-16T12:59:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:47.928 INFO:teuthology.orchestra.run.smithi042.stdout:1943 trunc from 0xbf80d1a to 0xac3cf6 2024-09-16T12:59:47.931 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:47.961 INFO:teuthology.orchestra.run.smithi042.stdout:1944 read 0xa07301 thru 0xa12e2b (0xbb2b bytes) 2024-09-16T12:59:47.962 INFO:teuthology.orchestra.run.smithi042.stdout:1946 read 0x774d73 thru 0x775d77 (0x1005 bytes) 2024-09-16T12:59:47.987 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:47.979+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:47.998 INFO:teuthology.orchestra.run.smithi042.stdout:1947 writesame 0xc2ffff4 thru 0xc30f74d data_size 0x7(0xf75a bytes) 2024-09-16T12:59:47.998 INFO:teuthology.orchestra.run.smithi042.stdout:1948 read 0x4ebc549 thru 0x4ec8de3 (0xc89b bytes) 2024-09-16T12:59:48.009 INFO:teuthology.orchestra.run.smithi042.stdout:1949 trunc from 0xc30f74e to 0x6686bb2 2024-09-16T12:59:48.022 INFO:teuthology.orchestra.run.smithi042.stdout:1950 punch from 0x3ca4f66 to 0x3ca6404, (0x149e bytes) 2024-09-16T12:59:48.022 INFO:teuthology.orchestra.run.smithi042.stdout:1951 read 0x234a78c thru 0x23586ee (0xdf63 bytes) 2024-09-16T12:59:48.022 INFO:teuthology.orchestra.run.smithi042.stdout:1952 write 0x682c98 thru 0x683613 (0x97c bytes) 2024-09-16T12:59:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:48.032 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:48.041 INFO:teuthology.orchestra.run.smithi042.stdout:1953 writesame 0x6c63403 thru 0x6c6aed8 data_size 0x9(0x7ad6 bytes) 2024-09-16T12:59:48.041 INFO:teuthology.orchestra.run.smithi042.stdout:1954 punch from 0x12b378c to 0x12bf127, (0xb99b bytes) 2024-09-16T12:59:48.041 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:48.041 INFO:teuthology.orchestra.run.smithi042.stdout:1956 punch from 0x6685477 to 0x6695456, (0xffdf bytes) 2024-09-16T12:59:48.047 INFO:teuthology.orchestra.run.smithi042.stdout:1957 trunc from 0x6c6aed9 to 0x8dadc4b 2024-09-16T12:59:48.059 INFO:teuthology.orchestra.run.smithi042.stdout:1958 read 0x7b6dc95 thru 0x7b76692 (0x89fe bytes) 2024-09-16T12:59:48.059 INFO:teuthology.orchestra.run.smithi042.stdout:1960 read 0x6187c6a thru 0x618a52f (0x28c6 bytes) 2024-09-16T12:59:48.059 INFO:teuthology.orchestra.run.smithi042.stdout:1962 read 0x2023174 thru 0x202d4ae (0xa33b bytes) 2024-09-16T12:59:48.059 INFO:teuthology.orchestra.run.smithi042.stdout:1963 write 0x70a9fe thru 0x719fb5 (0xf5b8 bytes) 2024-09-16T12:59:48.062 INFO:teuthology.orchestra.run.smithi042.stdout:1964 write 0x2d54e0d thru 0x2d5dd14 (0x8f08 bytes) 2024-09-16T12:59:48.068 INFO:teuthology.orchestra.run.smithi042.stdout:1965 punch from 0x24de3b9 to 0x24e1736, (0x337d bytes) 2024-09-16T12:59:48.068 INFO:teuthology.orchestra.run.smithi042.stdout:1967 punch from 0x53644e5 to 0x5369774, (0x528f bytes) 2024-09-16T12:59:48.087 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:48.079+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:48.088 INFO:teuthology.orchestra.run.smithi042.stdout:1968 writesame 0xc692e7a thru 0xc6974dd data_size 0x55(0x4664 bytes) 2024-09-16T12:59:48.113 INFO:teuthology.orchestra.run.smithi042.stdout:1969 trunc from 0xc6974de to 0x3d136c 2024-09-16T12:59:48.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:48.142 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:48.146 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:48.151 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:48.180 INFO:teuthology.orchestra.run.smithi042.stdout:1971 write 0xc74f269 thru 0xc75ef56 (0xfcee bytes) 2024-09-16T12:59:48.187 INFO:teuthology.orchestra.run.smithi042.stdout:1972 punch from 0xbfd259d to 0xbfe0c3b, (0xe69e bytes) 2024-09-16T12:59:48.187 INFO:teuthology.orchestra.run.smithi042.stdout:1973 read 0x260051d thru 0x260d275 (0xcd59 bytes) 2024-09-16T12:59:48.188 INFO:teuthology.orchestra.run.smithi042.stdout:1974 write 0x77e7283 thru 0x77f0d92 (0x9b10 bytes) 2024-09-16T12:59:48.188 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:48.182+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:1978 read 0x1c50120 thru 0x1c5b299 (0xb17a bytes) 2024-09-16T12:59:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:1981 writesame 0x3ffe605 thru 0x400c314 data_size 0x90(0xdd10 bytes) 2024-09-16T12:59:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:1982 write 0x50bb5e6 thru 0x50c644b (0xae66 bytes) 2024-09-16T12:59:48.211 INFO:teuthology.orchestra.run.smithi042.stdout:1983 write 0xb446b5 thru 0xb4f7be (0xb10a bytes) 2024-09-16T12:59:48.217 INFO:teuthology.orchestra.run.smithi042.stdout:1985 writesame 0xa7151c7 thru 0xa7209bf data_size 0x3(0xb7f9 bytes) 2024-09-16T12:59:48.237 INFO:teuthology.orchestra.run.smithi042.stdout:1987 write 0x41c3142 thru 0x41d29d5 (0xf894 bytes) 2024-09-16T12:59:48.243 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:48.243 INFO:teuthology.orchestra.run.smithi042.stdout:1989 read 0x7772b40 thru 0x777ca8a (0x9f4b bytes) 2024-09-16T12:59:48.252 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:48.252 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:48.261 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:48.266 INFO:teuthology.orchestra.run.smithi042.stdout:1990 trunc from 0xc75ef57 to 0x21aacfa 2024-09-16T12:59:48.288 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:48.288 INFO:teuthology.orchestra.run.smithi042.stdout:1992 write 0x15ece82 thru 0x15f2875 (0x59f4 bytes) 2024-09-16T12:59:48.293 INFO:teuthology.orchestra.run.smithi042.stdout:1993 write 0x4d8b63 thru 0x4d8eb9 (0x357 bytes) 2024-09-16T12:59:48.296 INFO:teuthology.orchestra.run.smithi042.stdout:1994 read 0x141d336 thru 0x141fad0 (0x279b bytes) 2024-09-16T12:59:48.296 INFO:teuthology.orchestra.run.smithi042.stdout:1995 read 0x7d6d8b thru 0x7d983a (0x2ab0 bytes) 2024-09-16T12:59:48.296 INFO:teuthology.orchestra.run.smithi042.stdout:1996 read 0x2adec0 thru 0x2af3d2 (0x1513 bytes) 2024-09-16T12:59:48.296 INFO:teuthology.orchestra.run.smithi042.stdout:1997 clone 16 order 19 su 8192 sc 9 2024-09-16T12:59:48.310 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:48.306+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:48.356 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:48.357 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:48.365 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:48.389 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:48.382+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:48.466 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:48.466 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:48.475 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:48.490 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:48.483+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:48.591 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:48.586+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:48.624 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:48.624 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:48.633 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:48.692 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:48.683+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:48.737 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:48.737 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:48.746 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:48.792 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:48.786+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:48.847 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:48.847 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:48.856 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:48.893 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:48.886+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:48.969 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:48.969 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:48.979 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:48.993 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:48.983+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:49.047 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T12:59:49.090 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:49.090 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:49.094 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:49.090+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:49.099 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:49.160 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone15 2024-09-16T12:59:49.195 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:49.187+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:49.205 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:49.206 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:49.221 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:49.282 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-16T12:59:49.296 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:49.287+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:49.365 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:49.365 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:49.376 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:49.422 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:49.415+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:49.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:49.498 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:49.491+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:49.573 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:49.573 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:49.583 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:49.599 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:49.594+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:49.649 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:49.700 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:49.691+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:49.712 INFO:teuthology.orchestra.run.smithi042.stdout:2001 trunc from 0x21aacfa to 0xa3d9957 2024-09-16T12:59:49.732 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:49.732 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:49.742 INFO:teuthology.orchestra.run.smithi042.stdout:2004 writesame 0xe62d09f thru 0xe62e9cc data_size 0xb(0x192e bytes) 2024-09-16T12:59:49.743 INFO:teuthology.orchestra.run.smithi042.stdout:2005 read 0xb447ab thru 0xb49a6f (0x52c5 bytes) 2024-09-16T12:59:49.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:49.745 INFO:teuthology.orchestra.run.smithi042.stdout:2006 write 0x5e4b9fd thru 0x5e57e29 (0xc42d bytes) 2024-09-16T12:59:49.759 INFO:teuthology.orchestra.run.smithi042.stdout:2009 writesame 0x5870e80 thru 0x5873ce4 data_size 0x25(0x2e65 bytes) 2024-09-16T12:59:49.766 INFO:teuthology.orchestra.run.smithi042.stdout:2010 punch from 0xdcc3316 to 0xdcc6245, (0x2f2f bytes) 2024-09-16T12:59:49.766 INFO:teuthology.orchestra.run.smithi042.stdout:2012 punch from 0xb13bfff to 0xb141a2a, (0x5a2b bytes) 2024-09-16T12:59:49.766 INFO:teuthology.orchestra.run.smithi042.stdout:2013 punch from 0x305bf57 to 0x3065bb6, (0x9c5f bytes) 2024-09-16T12:59:49.766 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:49.766 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:49.766 INFO:teuthology.orchestra.run.smithi042.stdout:2019 read 0x679a3dc thru 0x67a31ef (0x8e14 bytes) 2024-09-16T12:59:49.766 INFO:teuthology.orchestra.run.smithi042.stdout:2020 writesame 0xe032b2f thru 0xe03ad38 data_size 0x5(0x820a bytes) 2024-09-16T12:59:49.794 INFO:teuthology.orchestra.run.smithi042.stdout:2021 trunc from 0xe62e9cd to 0x1cc2c6f 2024-09-16T12:59:49.801 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:49.794+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:49.801 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:49.828 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:49.842 INFO:teuthology.orchestra.run.smithi042.stdout:2022 write 0x73b69c6 thru 0x73bca1f (0x605a bytes) 2024-09-16T12:59:49.853 INFO:teuthology.orchestra.run.smithi042.stdout:2024 read 0x255be79 thru 0x2560019 (0x41a1 bytes) 2024-09-16T12:59:49.853 INFO:teuthology.orchestra.run.smithi042.stdout:2025 write 0x4d1ea06 thru 0x4d2a34a (0xb945 bytes) 2024-09-16T12:59:49.859 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:49.859 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:49.859 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:49.867 INFO:teuthology.orchestra.run.smithi042.stdout:2026 trunc from 0x73bca20 to 0x75c3f85 2024-09-16T12:59:49.871 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:49.880 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:49.886 INFO:teuthology.orchestra.run.smithi042.stdout:2029 write 0xafc01ad thru 0xafce1c3 (0xe017 bytes) 2024-09-16T12:59:49.902 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:49.898+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:49.904 INFO:teuthology.orchestra.run.smithi042.stdout:2030 punch from 0x9f8d7db to 0x9f955b2, (0x7dd7 bytes) 2024-09-16T12:59:49.904 INFO:teuthology.orchestra.run.smithi042.stdout:2031 read 0x975e2b7 thru 0x976c173 (0xdebd bytes) 2024-09-16T12:59:49.904 INFO:teuthology.orchestra.run.smithi042.stdout:2032 write 0x4a67fed thru 0x4a773a2 (0xf3b6 bytes) 2024-09-16T12:59:49.916 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:49.918 INFO:teuthology.orchestra.run.smithi042.stdout:2034 write 0x3b496e8 thru 0x3b54b4d (0xb466 bytes) 2024-09-16T12:59:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:2035 punch from 0xadbcc21 to 0xadc4ec7, (0x82a6 bytes) 2024-09-16T12:59:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:2036 punch from 0x5bf3c09 to 0x5bf6ffc, (0x33f3 bytes) 2024-09-16T12:59:49.930 INFO:teuthology.orchestra.run.smithi042.stdout:2037 write 0x320ae1d thru 0x3218d2d (0xdf11 bytes) 2024-09-16T12:59:49.941 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:49.941 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:49.941 INFO:teuthology.orchestra.run.smithi042.stdout:2040 writesame 0x597e4f2 thru 0x598df75 data_size 0x2(0xfa84 bytes) 2024-09-16T12:59:49.950 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:49.960 INFO:teuthology.orchestra.run.smithi042.stdout:2041 punch from 0x4f4f2d8 to 0x4f5f143, (0xfe6b bytes) 2024-09-16T12:59:49.961 INFO:teuthology.orchestra.run.smithi042.stdout:2042 read 0x574968f thru 0x5754114 (0xaa86 bytes) 2024-09-16T12:59:49.961 INFO:teuthology.orchestra.run.smithi042.stdout:2043 punch from 0x4737dd0 to 0x473b31b, (0x354b bytes) 2024-09-16T12:59:49.962 INFO:teuthology.orchestra.run.smithi042.stdout:2044 punch from 0x5571ed5 to 0x557d482, (0xb5ad bytes) 2024-09-16T12:59:49.962 INFO:teuthology.orchestra.run.smithi042.stdout:2045 read 0x9027d61 thru 0x902a55d (0x27fd bytes) 2024-09-16T12:59:49.965 INFO:teuthology.orchestra.run.smithi042.stdout:2046 trunc from 0xafce1c4 to 0x8df9dc2 2024-09-16T12:59:49.973 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:49.979 INFO:teuthology.orchestra.run.smithi042.stdout:2047 write 0x862be79 thru 0x863adc1 (0xef49 bytes) 2024-09-16T12:59:49.995 INFO:teuthology.orchestra.run.smithi042.stdout:2048 read 0x6f5df69 thru 0x6f609ae (0x2a46 bytes) 2024-09-16T12:59:49.995 INFO:teuthology.orchestra.run.smithi042.stdout:2049 punch from 0x49ebbb9 to 0x49eefb8, (0x33ff bytes) 2024-09-16T12:59:49.995 INFO:teuthology.orchestra.run.smithi042.stdout:2050 punch from 0x22a4544 to 0x22b3ffb, (0xfab7 bytes) 2024-09-16T12:59:49.998 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:50.000 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:50.001 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:50.003 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:49.998+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:50.006 INFO:teuthology.orchestra.run.smithi042.stdout:2051 writesame 0xb3b0466 thru 0xb3ba230 data_size 0x5(0x9dcb bytes) 2024-09-16T12:59:50.006 INFO:teuthology.orchestra.run.smithi042.stdout:2053 read 0x99ee877 thru 0x99f709f (0x8829 bytes) 2024-09-16T12:59:50.008 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:50.008 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:50.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:50.008 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.224032524999998, 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003284908, 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 10 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779657+0000", 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1239999999999999e-06 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779667+0000", 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0487e-05 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779683+0000", 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6146000000000001e-05 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779695+0000", 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1347e-05 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779769+0000", 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.449e-05 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779918+0000", 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000148462 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.780088+0000", 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017066600000000001 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.781019+0000", 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093024399999999999 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782840+0000", 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018211379999999999 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782899+0000", 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8777000000000003e-05 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782940+0000", 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1026999999999999e-05 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:502 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 726006~2122] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.860264592, 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026083510000000001, 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 502 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143423+0000", 2024-09-16T12:59:50.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143426+0000", 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0460000000000001e-06 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143465+0000", 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143507+0000", 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1678999999999999e-05 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143523+0000", 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6464e-05 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143631+0000", 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107357 2024-09-16T12:59:50.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.143882+0000", 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025138100000000001 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.144130+0000", 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024822700000000001 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145527+0000", 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001396986 2024-09-16T12:59:50.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145925+0000", 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000397964 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.145985+0000", 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9896999999999999e-05 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.146031+0000", 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5938e-05 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:681 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [set-alloc-hint object_size 1048576 write_size 1048576,write 119408~11664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.395126455000003, 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035485299999999998, 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.020 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 681 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608564+0000", 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.146e-06 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608592+0000", 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8585000000000002e-05 2024-09-16T12:59:50.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608635+0000", 2024-09-16T12:59:50.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629000000000001e-05 2024-09-16T12:59:50.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608649+0000", 2024-09-16T12:59:50.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4270000000000001e-05 2024-09-16T12:59:50.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608797+0000", 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014800499999999999 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609162+0000", 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036455100000000001 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609411+0000", 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024895400000000002 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611008+0000", 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015968860000000001 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611948+0000", 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094050999999999998 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612013+0000", 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5185999999999996e-05 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612109+0000", 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5808000000000002e-05 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:688 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.359796828, 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027962989999999999, 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 688 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643890+0000", 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643893+0000", 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8200000000000001e-06 2024-09-16T12:59:50.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643906+0000", 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643953+0000", 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6399000000000001e-05 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.643972+0000", 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9130000000000001e-05 2024-09-16T12:59:50.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644163+0000", 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019125899999999999 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644426+0000", 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026297899999999999 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.644768+0000", 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000342574 2024-09-16T12:59:50.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646240+0000", 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001471312 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646617+0000", 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037757399999999999 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646658+0000", 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1083e-05 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.646687+0000", 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8248000000000001e-05 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:810 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 443751~14451] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.107042925000002, 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036011279999999999, 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 810 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896648+0000", 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6109999999999998e-06 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896701+0000", 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3168999999999999e-05 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896743+0000", 2024-09-16T12:59:50.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1445e-05 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896758+0000", 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5591e-05 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896878+0000", 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120256 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897013+0000", 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134327 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897221+0000", 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020804800000000001 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.899109+0000", 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018886899999999999 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900084+0000", 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097495899999999996 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900178+0000", 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3264999999999996e-05 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900245+0000", 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7767000000000004e-05 2024-09-16T12:59:50.037 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:826 4.4 e38/31 4:22c6e589:::rbd_data.11b59d02cc38.000000000000005a:head v 38'8, mlcod=37'6)", 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.073524370000001, 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028983849999999999, 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930163+0000", 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930165+0000", 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.114e-06 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930191+0000", 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6057000000000001e-05 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930222+0000", 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0959e-05 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930235+0000", 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930349+0000", 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011372200000000001 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.930402+0000", 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2766000000000003e-05 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.932954+0000", 2024-09-16T12:59:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025517980000000001 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.933061+0000", 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010770800000000001 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:860 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [set-alloc-hint object_size 1048576 write_size 1048576,write 997016~9314] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:50.042 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.005411797000001, 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037930170000000001, 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 860 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998280+0000", 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0969999999999999e-06 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998307+0000", 2024-09-16T12:59:50.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7804999999999999e-05 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998349+0000", 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1838000000000002e-05 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998364+0000", 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5194000000000001e-05 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998511+0000", 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014693900000000001 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998912+0000", 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000401066 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.999376+0000", 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000463938 2024-09-16T12:59:50.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001244+0000", 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018672389999999999 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001920+0000", 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067617100000000004 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002008+0000", 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8342000000000002e-05 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002068+0000", 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0387999999999998e-05 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:995 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.696530704000001, 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040928859999999996, 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 995 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307159+0000", 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9739999999999998e-06 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307174+0000", 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4134000000000001e-05 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307211+0000", 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7385000000000002e-05 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307230+0000", 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8961e-05 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307461+0000", 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230721 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307710+0000", 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024951200000000002 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308016+0000", 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030602199999999998 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309601+0000", 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015845200000000001 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311094+0000", 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00149321 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311180+0000", 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6142000000000003e-05 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311249+0000", 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9305000000000001e-05 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3724 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.092891892000001, 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002592624, 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910795+0000", 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910798+0000", 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7350000000000001e-06 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910813+0000", 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4469e-05 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910847+0000", 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3986999999999998e-05 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910869+0000", 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2764000000000001e-05 2024-09-16T12:59:50.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.910939+0000", 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0086000000000006e-05 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.911119+0000", 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179598 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.911351+0000", 2024-09-16T12:59:50.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023168200000000001 2024-09-16T12:59:50.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.912688+0000", 2024-09-16T12:59:50.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013376729999999999 2024-09-16T12:59:50.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913294+0000", 2024-09-16T12:59:50.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060521299999999995 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913343+0000", 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9537999999999999e-05 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.913388+0000", 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4879000000000002e-05 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4040 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.541362170999999, 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026268979999999999, 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4040 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462325+0000", 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462327+0000", 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0229999999999999e-06 2024-09-16T12:59:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462340+0000", 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3189e-05 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462373+0000", 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2555000000000003e-05 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462386+0000", 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2935000000000001e-05 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462458+0000", 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1847999999999994e-05 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.462582+0000", 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012399199999999999 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.463055+0000", 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047307700000000002 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464541+0000", 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014867649999999999 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464841+0000", 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029975500000000002 2024-09-16T12:59:50.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464912+0000", 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0344000000000003e-05 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.464952+0000", 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0414999999999998e-05 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4042 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.535000868000001, 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002527208, 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4042 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468686+0000", 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468688+0000", 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9549999999999999e-06 2024-09-16T12:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468700+0000", 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1443e-05 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468726+0000", 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6591999999999999e-05 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468737+0000", 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0951999999999999e-05 2024-09-16T12:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468841+0000", 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010335 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.468995+0000", 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015392800000000001 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.469556+0000", 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00056112700000000002 2024-09-16T12:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471063+0000", 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001507226 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471130+0000", 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6693999999999995e-05 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471176+0000", 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6223000000000001e-05 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.471214+0000", 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7718e-05 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4044 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.529233155, 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002965217, 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4044 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474456+0000", 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.818e-06 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474465+0000", 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5459999999999997e-06 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474490+0000", 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5058e-05 2024-09-16T12:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474502+0000", 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1110999999999999e-05 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474610+0000", 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108606 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474815+0000", 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020453800000000001 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.475681+0000", 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086620700000000002 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477079+0000", 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001398388 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477340+0000", 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026078800000000002 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477385+0000", 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5014e-05 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477419+0000", 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4143000000000002e-05 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4069 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.467858250999999, 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025697279999999999, 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4069 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535829+0000", 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535831+0000", 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8109999999999999e-06 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535841+0000", 2024-09-16T12:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0267e-05 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535862+0000", 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0693e-05 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535869+0000", 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7309999999999993e-06 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.535958+0000", 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8721000000000007e-05 2024-09-16T12:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.536094+0000", 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013532999999999999 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.536561+0000", 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000467057 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538128+0000", 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001567337 2024-09-16T12:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538285+0000", 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015667900000000001 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538355+0000", 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9991000000000001e-05 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.538399+0000", 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4110999999999998e-05 2024-09-16T12:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:5580 4.12 e46/40 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head v 46'11, mlcod=46'9)", 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.294693023000001, 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039131280000000001, 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708997+0000", 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4109999999999999e-06 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710732+0000", 2024-09-16T12:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001735002 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710766+0000", 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-05 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710779+0000", 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3137e-05 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710905+0000", 2024-09-16T12:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012594799999999999 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710972+0000", 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7062000000000003e-05 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.712815+0000", 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018428419999999999 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.712907+0000", 2024-09-16T12:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2516000000000007e-05 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6800 4.1c 4:3d97daaf:::rbd_data.11b58f77e1c4.0000000000000030:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1985195~61789] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:32.580187+0000", 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.423500013999998, 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002873221, 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6800 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580187+0000", 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580187+0000", 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580190+0000", 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8880000000000001e-06 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580275+0000", 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4672000000000005e-05 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580313+0000", 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7991000000000003e-05 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580330+0000", 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7425e-05 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580416+0000", 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5928999999999997e-05 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580627+0000", 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021073899999999999 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.580877+0000", 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024978100000000003 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.582227+0000", 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013502729999999999 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.582955+0000", 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072785299999999999 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.583015+0000", 2024-09-16T12:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0149999999999998e-05 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.583060+0000", 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5519999999999998e-05 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:6806 4.1c 4:3d97daaf:::rbd_data.11b58f77e1c4.0000000000000030:head [stat,truncate 2002817] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:32.608984+0000", 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.394702795000001, 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027044930000000001, 2024-09-16T12:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6806 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.608984+0000", 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.608984+0000", 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.608989+0000", 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2359999999999998e-06 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609009+0000", 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0757000000000001e-05 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609069+0000", 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0019000000000003e-05 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609094+0000", 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4119999999999999e-05 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609229+0000", 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000135773 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609396+0000", 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000166501 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.609647+0000", 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025091499999999999 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611408+0000", 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017616019999999999 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611590+0000", 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181192 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611643+0000", 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3368000000000001e-05 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:32.611689+0000", 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6010000000000002e-05 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7104 4.1c 4:394c1983:::rbd_data.11b58f77e1c4.0000000000000024:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.957177860000002, 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031870290000000001, 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7104 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046511+0000", 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9819999999999998e-06 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046518+0000", 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4489999999999996e-06 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046536+0000", 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8544999999999999e-05 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046544+0000", 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.729e-06 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046624+0000", 2024-09-16T12:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9708000000000003e-05 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046755+0000", 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130846 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049696+0000", 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029417699999999998 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:8852 4.d e55/31 4:b280c692:::rbd_data.11b55eb65e78.0000000000000004:head v 55'43, mlcod=54'40)", 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:38.090571+0000", 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.91311574, 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002909961, 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090571+0000", 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090571+0000", 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090573+0000", 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.274e-06 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090588+0000", 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5109999999999999e-05 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090613+0000", 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4916999999999999e-05 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090625+0000", 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2354000000000001e-05 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090760+0000", 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134468 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.090826+0000", 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6765000000000002e-05 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.093373+0000", 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025467110000000001 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:38.093481+0000", 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108362 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15008 4.1c 4:3e8625c6:::rbd_data.11b5c2c64266.0000000000000162:head [set-alloc-hint object_size 524288 write_size 524288,write 32768~452] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:49.894991+0000", 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.10869596300000001, 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041292409999999996, 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15008 2024-09-16T12:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.894991+0000", 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.894991+0000", 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.894994+0000", 2024-09-16T12:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895004+0000", 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0373e-05 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895028+0000", 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4193999999999999e-05 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895039+0000", 2024-09-16T12:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0268999999999999e-05 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895119+0000", 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0209000000000004e-05 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895289+0000", 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000169779 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895402+0000", 2024-09-16T12:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113443 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.897278+0000", 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001876305 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.899011+0000", 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017325299999999999 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.899070+0000", 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9426999999999999e-05 2024-09-16T12:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.899120+0000", 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0251999999999999e-05 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15071 4.1c 4:3ad94978:::rbd_data.11b5c2c64266.00000000000000ad:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:49.944671+0000", 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.059016506000000003, 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031721560000000002, 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15071 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944671+0000", 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944671+0000", 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944673+0000", 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.153e-06 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944682+0000", 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1230000000000006e-06 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944707+0000", 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4746999999999999e-05 2024-09-16T12:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944716+0000", 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4380000000000001e-06 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944787+0000", 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0776000000000001e-05 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944954+0000", 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016750000000000001 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.945648+0000", 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00069389400000000002 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.947711+0000", 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020623659999999999 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.947773+0000", 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2126999999999997e-05 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.947806+0000", 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3277e-05 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.947843+0000", 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6755000000000003e-05 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:2054 write 0xd9642b2 thru 0xd96da87 (0x97d6 bytes) 2024-09-16T12:59:50.118 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:2055 trunc from 0xd96da88 to 0x4a861e5 2024-09-16T12:59:50.119 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T12:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:2056 read 0x2f43b50 thru 0x2f44e83 (0x1334 bytes) 2024-09-16T12:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:2057 writesame 0x4769bfe thru 0x47753c5 data_size 0x4(0xb7c8 bytes) 2024-09-16T12:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:2058 write 0x2abaf02 thru 0x2ac7b4c (0xcc4b bytes) 2024-09-16T12:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:2059 writesame 0xcb7a24 thru 0xcb8bf8 data_size 0xb(0x11d5 bytes) 2024-09-16T12:59:50.120 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:50.107+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:2060 read 0xa4a1a6 thru 0xa590f4 (0xef4f bytes) 2024-09-16T12:59:50.127 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T12:59:50.127 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T12:59:50.127 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T12:59:50.127 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T12:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:2061 write 0xc30d5d5 thru 0xc3163ac (0x8dd8 bytes) 2024-09-16T12:59:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:2063 write 0xe97c0a6 thru 0xe984364 (0x82bf bytes) 2024-09-16T12:59:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:2064 read 0x5ae5557 thru 0x5af234a (0xcdf4 bytes) 2024-09-16T12:59:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:2066 read 0xdd80623 thru 0xdd8a54f (0x9f2d bytes) 2024-09-16T12:59:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:2068 punch from 0x3ba9ba7 to 0x3bb2fe7, (0x9440 bytes) 2024-09-16T12:59:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:2069 punch from 0x34d50a0 to 0x34ddf0f, (0x8e6f bytes) 2024-09-16T12:59:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:2071 write 0x194983b thru 0x194cd98 (0x355e bytes) 2024-09-16T12:59:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:2073 read 0x7bed848 thru 0x7bf69f9 (0x91b2 bytes) 2024-09-16T12:59:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:2074 punch from 0xc0790b4 to 0xc07ad41, (0x1c8d bytes) 2024-09-16T12:59:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:2076 punch from 0xe4ede4c to 0xe4fab2c, (0xcce0 bytes) 2024-09-16T12:59:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:2077 write 0x72583f0 thru 0x725e35e (0x5f6f bytes) 2024-09-16T12:59:50.206 INFO:teuthology.orchestra.run.smithi042.stdout:2078 read 0x6252728 thru 0x625879b (0x6074 bytes) 2024-09-16T12:59:50.206 INFO:teuthology.orchestra.run.smithi042.stdout:2081 writesame 0x92ef5fd thru 0x92fd2ab data_size 0x1(0xdcaf bytes) 2024-09-16T12:59:50.218 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:50.210+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:50.229 INFO:teuthology.orchestra.run.smithi042.stdout:2083 write 0x52616df thru 0x52707c2 (0xf0e4 bytes) 2024-09-16T12:59:50.244 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.244 INFO:teuthology.orchestra.run.smithi042.stdout:2089 write 0xac18679 thru 0xac198c1 (0x1249 bytes) 2024-09-16T12:59:50.248 INFO:teuthology.orchestra.run.smithi042.stdout:2090 read 0x87f431c thru 0x87f56d0 (0x13b5 bytes) 2024-09-16T12:59:50.248 INFO:teuthology.orchestra.run.smithi042.stdout:2091 punch from 0x562ba28 to 0x5633385, (0x795d bytes) 2024-09-16T12:59:50.248 INFO:teuthology.orchestra.run.smithi042.stdout:2092 read 0x51311d4 thru 0x513a8f2 (0x971f bytes) 2024-09-16T12:59:50.249 INFO:teuthology.orchestra.run.smithi042.stdout:2093 writesame 0xc347e0d thru 0xc34b827 data_size 0x19(0x3a1b bytes) 2024-09-16T12:59:50.259 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:50.259 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:50.267 INFO:teuthology.orchestra.run.smithi042.stdout:2094 trunc from 0xe984365 to 0x43cea2f 2024-09-16T12:59:50.275 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:50.301 INFO:teuthology.orchestra.run.smithi042.stdout:2095 punch from 0x2bf7419 to 0x2bf7d6e, (0x955 bytes) 2024-09-16T12:59:50.301 INFO:teuthology.orchestra.run.smithi042.stdout:2097 read 0x3e9ab42 thru 0x3ea8b3a (0xdff9 bytes) 2024-09-16T12:59:50.301 INFO:teuthology.orchestra.run.smithi042.stdout:2098 write 0x14504f3 thru 0x146010a (0xfc18 bytes) 2024-09-16T12:59:50.347 INFO:teuthology.orchestra.run.smithi042.stdout:2099 write 0xb0f1d9b thru 0xb0fd918 (0xbb7e bytes) 2024-09-16T12:59:50.347 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:50.339+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:50.362 INFO:teuthology.orchestra.run.smithi042.stdout:2100 read 0x4b6fec6 thru 0x4b71853 (0x198e bytes) 2024-09-16T12:59:50.362 INFO:teuthology.orchestra.run.smithi042.stdout:2101 write 0x51ab7f3 thru 0x51b27b1 (0x6fbf bytes) 2024-09-16T12:59:50.372 INFO:teuthology.orchestra.run.smithi042.stdout:2102 write 0x1d790c1 thru 0x1d79603 (0x543 bytes) 2024-09-16T12:59:50.374 INFO:teuthology.orchestra.run.smithi042.stdout:2103 read 0x361326 thru 0x36899d (0x7678 bytes) 2024-09-16T12:59:50.376 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.376 INFO:teuthology.orchestra.run.smithi042.stdout:2107 writesame 0x5de8e04 thru 0x5df4778 data_size 0x1(0xb975 bytes) 2024-09-16T12:59:50.400 INFO:teuthology.orchestra.run.smithi042.stdout:2108 read 0x146ffc5 thru 0x14755db (0x5617 bytes) 2024-09-16T12:59:50.410 INFO:teuthology.orchestra.run.smithi042.stdout:2109 write 0xbb6ae31 thru 0xbb7a91d (0xfaed bytes) 2024-09-16T12:59:50.420 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:50.411+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:50.425 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:50.426 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:50.426 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.426 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.426 INFO:teuthology.orchestra.run.smithi042.stdout:2114 punch from 0x3bd756d to 0x3be344c, (0xbedf bytes) 2024-09-16T12:59:50.436 INFO:teuthology.orchestra.run.smithi042.stdout:2116 write 0xcda6dd4 thru 0xcdab2da (0x4507 bytes) 2024-09-16T12:59:50.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:50.443 INFO:teuthology.orchestra.run.smithi042.stdout:2117 read 0x9a14b21 thru 0x9a1f154 (0xa634 bytes) 2024-09-16T12:59:50.443 INFO:teuthology.orchestra.run.smithi042.stdout:2118 read 0x71f7ab3 thru 0x71f93a0 (0x18ee bytes) 2024-09-16T12:59:50.443 INFO:teuthology.orchestra.run.smithi042.stdout:2120 writesame 0x34330a7 thru 0x343a0dc data_size 0x35(0x7036 bytes) 2024-09-16T12:59:50.448 INFO:teuthology.orchestra.run.smithi042.stdout:2123 punch from 0x206b8c6 to 0x206ca8b, (0x11c5 bytes) 2024-09-16T12:59:50.448 INFO:teuthology.orchestra.run.smithi042.stdout:2124 writesame 0xc8f0969 thru 0xc8f9047 data_size 0x11(0x86df bytes) 2024-09-16T12:59:50.459 INFO:teuthology.orchestra.run.smithi042.stdout:2125 write 0x2fe574 thru 0x301489 (0x2f16 bytes) 2024-09-16T12:59:50.489 INFO:teuthology.orchestra.run.smithi042.stdout:2126 trunc from 0xcdab2db to 0x5bddb80 2024-09-16T12:59:50.518 INFO:teuthology.orchestra.run.smithi042.stdout:2128 write 0x5c61d1e thru 0x5c7063f (0xe922 bytes) 2024-09-16T12:59:50.520 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:50.514+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:50.524 INFO:teuthology.orchestra.run.smithi042.stdout:2130 write 0x4351bb2 thru 0x435b7cf (0x9c1e bytes) 2024-09-16T12:59:50.528 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:59:50.528 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 65, 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7", 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-16T12:58:33.966359+0000", 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-16T12:59:50.039550+0000", 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-16T12:58:44.530413+0000", 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-16T12:59:39.968761+0000", 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5806080, 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "noscrub", 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-16T12:59:50.529 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 4, 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "quincy", 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-16T12:59:50.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:58:44.284177+0000", 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T12:59:50.531 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.532 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "17", 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-16T12:59:50.533 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.534 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.535 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:58:51.733915+0000", 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T12:59:50.536 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T12:59:50.537 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "26", 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "26", 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T12:59:50.538 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 3, 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 24, 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [ 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: 3 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": 3, 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": 3, 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-16T12:59:50.539 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-16T12:59:50.540 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-16T12:59:50.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "cache", 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:59:00.193651+0000", 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8201, 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,incomplete_clones,selfmanaged_snaps", 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T12:59:50.542 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 4, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 4, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 4, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 4, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 4, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T12:59:50.543 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "30", 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "26", 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 3, 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 24, 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T12:59:50.544 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": 2, 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "writeback", 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 250, 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T12:59:50.545 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bloom", 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "false_positive_probability": 0.050000000000000003, 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "target_size": 0, 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "seed": 0 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 60, 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 8, 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T12:59:50.546 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [ 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-16T12:59:50.547 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 4, 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:59:09.779825+0000", 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T12:59:50.548 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T12:59:50.549 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "64", 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 19, 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 64, 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T12:59:50.550 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-16T12:59:50.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-16T12:59:50.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "0d8a8277-07cd-40f1-8c98-98b8b51d1689", 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 57, 2024-09-16T12:59:50.553 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.556 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6819/22186", 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6823/22186", 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6830/22186", 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6826/22186", 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "6ce5996a-5e0e-4c9d-b268-27b34a14d830", 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T12:59:50.558 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 57, 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-16T12:59:50.559 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T12:59:50.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.560 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-16T12:59:50.560 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T12:59:50.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.560 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.560 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.561 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T12:59:50.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/22184", 2024-09-16T12:59:50.563 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/22184", 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/22184", 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/22184", 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "156042ab-909e-4368-b510-84018cb4354f", 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T12:59:50.564 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 57, 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T12:59:50.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/22185", 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/22185", 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/22185", 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/22185", 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b6c53823-446f-4413-a2ff-d2ebe8fe9c71", 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 57, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T12:59:50.570 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.573 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6818/22187", 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6822/22187", 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/22187", 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6827/22187", 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T12:59:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "278bfad1-010b-494e-ba61-bdf3935ca078", 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T12:59:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 57, 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6816", 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6817", 2024-09-16T12:59:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6818", 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6819", 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6822", 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6823", 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6820", 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6821", 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6817/22339", 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6819/22339", 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6823/22339", 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6821/22339", 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-16T12:59:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "ed1af73b-b108-4189-8223-f017e634e3e8", 2024-09-16T12:59:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T12:59:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 57, 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6808", 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6809", 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.583 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6810", 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6811", 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6814", 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6815", 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6812", 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6813", 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6809/22340", 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6811/22340", 2024-09-16T12:59:50.586 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6815/22340", 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6813/22340", 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "ce904ba8-02e9-4579-9635-b4b6c6de1a08", 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T12:59:50.587 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 60, 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6824", 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.588 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6825", 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6826", 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.589 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6827", 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6830", 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.590 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6831", 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6828", 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6829", 2024-09-16T12:59:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6825/22364", 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6827/22364", 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6831/22364", 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6829/22364", 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.592 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d911da1b-2725-46a8-bbfa-8968de47271f", 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 57, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6800", 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6801", 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6802", 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6803", 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.595 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6806", 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6807", 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6804", 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6805", 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6801/22341", 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6803/22341", 2024-09-16T12:59:50.597 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6807/22341", 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6805/22341", 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T12:59:50.598 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.264463+0000", 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.214474+0000", 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.217927+0000", 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T12:59:50.600 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.435822+0000", 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.451286+0000", 2024-09-16T12:59:50.601 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.287403+0000", 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.602 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.440296+0000", 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T12:59:50.603 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.140630+0000", 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-16T12:59:50.604 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-16T12:59:50.605 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-16T12:59:50.606 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-16T12:59:50.606 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-16T12:59:50.606 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-16T12:59:50.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.606 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:50.607 INFO:tasks.thrashosds.thrasher:j is {'epoch': 65, 'fsid': 'c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7', 'created': '2024-09-16T12:58:33.966359+0000', 'modified': '2024-09-16T12:59:50.039550+0000', 'last_up_change': '2024-09-16T12:58:44.530413+0000', 'last_in_change': '2024-09-16T12:59:39.968761+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T12:58:44.284177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T12:58:51.733915+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T12:59:00.193651+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T12:59:09.779825+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 64, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': '0d8a8277-07cd-40f1-8c98-98b8b51d1689', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6817', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 22186}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6821', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 22186}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6830', 'nonce': 22186}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6825', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6826', 'nonce': 22186}]}, 'public_addr': '172.21.15.42:6819/22186', 'cluster_addr': '172.21.15.42:6823/22186', 'heartbeat_back_addr': '172.21.15.42:6830/22186', 'heartbeat_front_addr': '172.21.15.42:6826/22186', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6ce5996a-5e0e-4c9d-b268-27b34a14d830', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 22184}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 22184}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 22184}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 22184}]}, 'public_addr': '172.21.15.42:6809/22184', 'cluster_addr': '172.21.15.42:6811/22184', 'heartbeat_back_addr': '172.21.15.42:6815/22184', 'heartbeat_front_addr': '172.21.15.42:6813/22184', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '156042ab-909e-4368-b510-84018cb4354f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 22185}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 22185}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 22185}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 22185}]}, 'public_addr': '172.21.15.42:6801/22185', 'cluster_addr': '172.21.15.42:6803/22185', 'heartbeat_back_addr': '172.21.15.42:6807/22185', 'heartbeat_front_addr': '172.21.15.42:6805/22185', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b6c53823-446f-4413-a2ff-d2ebe8fe9c71', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6818', 'nonce': 22187}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6822', 'nonce': 22187}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6829', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 22187}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 22187}]}, 'public_addr': '172.21.15.42:6818/22187', 'cluster_addr': '172.21.15.42:6822/22187', 'heartbeat_back_addr': '172.21.15.42:6831/22187', 'heartbeat_front_addr': '172.21.15.42:6827/22187', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '278bfad1-010b-494e-ba61-bdf3935ca078', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6816', 'nonce': 22339}, {'type': 'v1', 'addr': '172.21.15.136:6817', 'nonce': 22339}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6818', 'nonce': 22339}, {'type': 'v1', 'addr': '172.21.15.136:6819', 'nonce': 22339}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6822', 'nonce': 22339}, {'type': 'v1', 'addr': '172.21.15.136:6823', 'nonce': 22339}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6820', 'nonce': 22339}, {'type': 'v1', 'addr': '172.21.15.136:6821', 'nonce': 22339}]}, 'public_addr': '172.21.15.136:6817/22339', 'cluster_addr': '172.21.15.136:6819/22339', 'heartbeat_back_addr': '172.21.15.136:6823/22339', 'heartbeat_front_addr': '172.21.15.136:6821/22339', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ed1af73b-b108-4189-8223-f017e634e3e8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6809', 'nonce': 22340}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6810', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6811', 'nonce': 22340}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6814', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6815', 'nonce': 22340}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6813', 'nonce': 22340}]}, 'public_addr': '172.21.15.136:6809/22340', 'cluster_addr': '172.21.15.136:6811/22340', 'heartbeat_back_addr': '172.21.15.136:6815/22340', 'heartbeat_front_addr': '172.21.15.136:6813/22340', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ce904ba8-02e9-4579-9635-b4b6c6de1a08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6824', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.136:6825', 'nonce': 22364}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6826', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.136:6827', 'nonce': 22364}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6830', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.136:6831', 'nonce': 22364}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6828', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.136:6829', 'nonce': 22364}]}, 'public_addr': '172.21.15.136:6825/22364', 'cluster_addr': '172.21.15.136:6827/22364', 'heartbeat_back_addr': '172.21.15.136:6831/22364', 'heartbeat_front_addr': '172.21.15.136:6829/22364', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd911da1b-2725-46a8-bbfa-8968de47271f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6801', 'nonce': 22341}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6803', 'nonce': 22341}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6807', 'nonce': 22341}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6805', 'nonce': 22341}]}, 'public_addr': '172.21.15.136:6801/22341', 'cluster_addr': '172.21.15.136:6803/22341', 'heartbeat_back_addr': '172.21.15.136:6807/22341', 'heartbeat_front_addr': '172.21.15.136:6805/22341', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.264463+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.214474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.217927+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.435822+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.451286+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.287403+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.440296+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.140630+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T12:59:50.608 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T12:59:50.608 INFO:teuthology.orchestra.run.smithi042.stdout:2131 writesame 0x9965441 thru 0x9965c00 data_size 0x1f(0x7c0 bytes) 2024-09-16T12:59:50.608 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.609 INFO:teuthology.orchestra.run.smithi042.stdout:2133 writesame 0x9488f1b thru 0x94962b1 data_size 0x1(0xd397 bytes) 2024-09-16T12:59:50.609 INFO:teuthology.orchestra.run.smithi042.stdout:2135 read 0x417d597 thru 0x417dbc5 (0x62f bytes) 2024-09-16T12:59:50.609 INFO:teuthology.orchestra.run.smithi042.stdout:2136 writesame 0x1523cd6 thru 0x152f61e data_size 0x3(0xb949 bytes) 2024-09-16T12:59:50.610 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:50.610 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:50.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:50.610 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:50.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.610 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8579 4.11 4:895e0e12:::rbd_data.11b55eb65e78.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:50.610 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.742439+0000", 2024-09-16T12:59:50.610 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.835298761000001, 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025774687000000001, 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8579 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742439+0000", 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742439+0000", 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742443+0000", 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.473e-06 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742461+0000", 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8329999999999999e-05 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742504+0000", 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3098e-05 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742520+0000", 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5982999999999999e-05 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742635+0000", 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000114537 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742782+0000", 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014705100000000001 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.749390+0000", 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066079370000000004 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.767807+0000", 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018417086999999999 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.767925+0000", 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011807300000000001 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.768098+0000", 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172876 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.768214+0000", 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116242 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8611 4.11 4:895e0e12:::rbd_data.11b55eb65e78.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.815402+0000", 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.762336232999999, 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927149, 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8611 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815402+0000", 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815402+0000", 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815405+0000", 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8030000000000001e-06 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815416+0000", 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1741e-05 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815463+0000", 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6303e-05 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815474+0000", 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1189000000000001e-05 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815580+0000", 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010592899999999999 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815765+0000", 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185095 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.821526+0000", 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057609820000000004 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845015+0000", 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023488841999999999 2024-09-16T12:59:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845048+0000", 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2913999999999997e-05 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845204+0000", 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015631600000000001 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845329+0000", 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012503500000000001 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:13818 4.11 4:8dc8e4d5:::rbd_data.11b51cc7997f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:45.688547+0000", 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.8891907999999997, 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.032774062999999999, 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.622 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 13818 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.688547+0000", 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.688547+0000", 2024-09-16T12:59:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.688550+0000", 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5799999999999999e-06 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.689948+0000", 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013981040000000001 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.689981+0000", 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3479999999999998e-05 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.689994+0000", 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2286000000000001e-05 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.690084+0000", 2024-09-16T12:59:50.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9818e-05 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.690297+0000", 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021293899999999999 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.701762+0000", 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011465145 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721206+0000", 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019444110000000001 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721217+0000", 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1452e-05 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721263+0000", 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6097999999999998e-05 2024-09-16T12:59:50.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721321+0000", 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8050999999999997e-05 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15366 4.14 4:2baea869:::rbd_data.11b5c2c64266.0000000000000096:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.267941+0000", 2024-09-16T12:59:50.628 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.30979756600000002, 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015615755, 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15366 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267941+0000", 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267941+0000", 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267942+0000", 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7150000000000001e-06 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267950+0000", 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3320000000000003e-06 2024-09-16T12:59:50.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267974+0000", 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4618e-05 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267984+0000", 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3840000000000003e-06 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.268045+0000", 2024-09-16T12:59:50.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1011999999999998e-05 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.268133+0000", 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8713000000000003e-05 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.268230+0000", 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6569000000000003e-05 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269729+0000", 2024-09-16T12:59:50.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001499268 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.283483+0000", 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013754096 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.283525+0000", 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1835999999999997e-05 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.283556+0000", 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1211999999999999e-05 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15370 4.14 4:2d180289:::rbd_data.11b5c2c64266.00000000000000a3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.269714+0000", 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.30802423499999998, 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013915235999999999, 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15370 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269714+0000", 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269714+0000", 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269716+0000", 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.99e-06 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269724+0000", 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1659999999999994e-06 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269742+0000", 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7655e-05 2024-09-16T12:59:50.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269751+0000", 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.952e-06 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269801+0000", 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0481e-05 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269889+0000", 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7828e-05 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269994+0000", 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104885 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.271076+0000", 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001082216 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.283566+0000", 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012489591 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.283590+0000", 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4099000000000001e-05 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.283629+0000", 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9372999999999997e-05 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15507 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302464+0000", 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.275274458, 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013620312000000001, 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15507 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302464+0000", 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302464+0000", 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302464+0000", 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9800000000000003e-07 2024-09-16T12:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302468+0000", 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5470000000000001e-06 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302476+0000", 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5979999999999997e-06 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302481+0000", 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4850000000000003e-06 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.315958+0000", 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013477312999999999 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.316019+0000", 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0720000000000001e-05 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.316084+0000", 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5050999999999998e-05 2024-09-16T12:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15508 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 557056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302502+0000", 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27523588599999999, 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013724502, 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15508 2024-09-16T12:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302502+0000", 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302502+0000", 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302503+0000", 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4e-07 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302507+0000", 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7969999999999998e-06 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302518+0000", 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302523+0000", 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.463e-06 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.316107+0000", 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013584081 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.316172+0000", 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5183000000000003e-05 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.316227+0000", 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4409e-05 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15509 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302530+0000", 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.275208377, 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013832621999999999, 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15509 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302530+0000", 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302530+0000", 2024-09-16T12:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302530+0000", 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4099999999999999e-07 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302533+0000", 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1389999999999999e-06 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302542+0000", 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5529999999999993e-06 2024-09-16T12:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302547+0000", 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8110000000000002e-06 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.316243+0000", 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013696155999999999 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.316289+0000", 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6430000000000001e-05 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.316362+0000", 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2991999999999995e-05 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15510 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 704512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302552+0000", 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27518659299999998, 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014458164000000001, 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15510 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302552+0000", 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302552+0000", 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302552+0000", 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2099999999999997e-07 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302555+0000", 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.853e-06 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302563+0000", 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8399999999999995e-06 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302567+0000", 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-06 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.316386+0000", 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013818222999999999 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.316450+0000", 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4292e-05 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317010+0000", 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00055981499999999997 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15511 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302573+0000", 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27516511399999999, 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014626780000000001, 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15511 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302573+0000", 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302573+0000", 2024-09-16T12:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302574+0000", 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5400000000000001e-07 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302576+0000", 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9009999999999998e-06 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302585+0000", 2024-09-16T12:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4600000000000003e-06 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302590+0000", 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.865e-06 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317058+0000", 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014468391000000001 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317126+0000", 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7376e-05 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317200+0000", 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4233000000000006e-05 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15512 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302610+0000", 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27512806000000001, 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014758304, 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15512 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302610+0000", 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302610+0000", 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302611+0000", 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0699999999999996e-07 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302615+0000", 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2760000000000002e-06 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302625+0000", 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302631+0000", 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6069999999999998e-06 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317229+0000", 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014598076 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317298+0000", 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9243000000000001e-05 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317368+0000", 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0105999999999996e-05 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15513 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302637+0000", 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27510109399999999, 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014869713999999999, 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15513 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302637+0000", 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302637+0000", 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302638+0000", 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7700000000000004e-07 2024-09-16T12:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302640+0000", 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302648+0000", 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5870000000000004e-06 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302652+0000", 2024-09-16T12:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1559999999999997e-06 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317393+0000", 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014740567 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317455+0000", 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2335000000000005e-05 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317507+0000", 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1816e-05 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15514 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302657+0000", 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27508112800000001, 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015027508, 2024-09-16T12:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15514 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302657+0000", 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302657+0000", 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302658+0000", 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2e-07 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302660+0000", 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4119999999999999e-06 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302668+0000", 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2449999999999998e-06 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302673+0000", 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7339999999999999e-06 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317525+0000", 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014852547000000001 2024-09-16T12:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317599+0000", 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3670000000000004e-05 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317685+0000", 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5379999999999999e-05 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15515 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302678+0000", 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27505996100000002, 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015210016999999999, 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15515 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302678+0000", 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302678+0000", 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302679+0000", 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.06e-07 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302681+0000", 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.655e-06 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302689+0000", 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7759999999999997e-06 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302694+0000", 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8110000000000002e-06 2024-09-16T12:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317715+0000", 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015021435 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317801+0000", 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5972999999999998e-05 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317888+0000", 2024-09-16T12:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6860999999999994e-05 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15516 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302699+0000", 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27503906500000003, 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015400624, 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15516 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302699+0000", 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302699+0000", 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302699+0000", 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302702+0000", 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3520000000000001e-06 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302708+0000", 2024-09-16T12:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7130000000000002e-06 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302710+0000", 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7410000000000001e-06 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317919+0000", 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01520839 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317998+0000", 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8959000000000001e-05 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318100+0000", 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010204699999999999 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15517 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 401408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302713+0000", 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27502504100000003, 2024-09-16T12:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015596560000000001, 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15517 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302713+0000", 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302713+0000", 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302713+0000", 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302715+0000", 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6279999999999999e-06 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302719+0000", 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8959999999999996e-06 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302721+0000", 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5119999999999998e-06 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318131+0000", 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015409849999999999 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318214+0000", 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2273000000000003e-05 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318310+0000", 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6068000000000002e-05 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15518 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302724+0000", 2024-09-16T12:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27501419199999999, 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015782292, 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15518 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302724+0000", 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302724+0000", 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302724+0000", 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.41e-07 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302726+0000", 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.42e-06 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302730+0000", 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302732+0000", 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318345+0000", 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015612475000000001 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318423+0000", 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8694000000000002e-05 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318506+0000", 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2922000000000003e-05 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15519 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302738+0000", 2024-09-16T12:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.27500035099999998, 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015930841000000001, 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15519 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302738+0000", 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302738+0000", 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302738+0000", 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302740+0000", 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302748+0000", 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8460000000000008e-06 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302752+0000", 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4710000000000001e-06 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318542+0000", 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015789257000000001 2024-09-16T12:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318615+0000", 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3313999999999996e-05 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318669+0000", 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3693000000000002e-05 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15520 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302759+0000", 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.274978891, 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016018316000000001, 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15520 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302759+0000", 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302759+0000", 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302760+0000", 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5300000000000004e-07 2024-09-16T12:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302762+0000", 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6570000000000001e-06 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302771+0000", 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2770000000000005e-06 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302774+0000", 2024-09-16T12:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.92e-06 2024-09-16T12:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318688+0000", 2024-09-16T12:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015913882000000001 2024-09-16T12:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318730+0000", 2024-09-16T12:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2710999999999998e-05 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318778+0000", 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7315999999999999e-05 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15521 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302805+0000", 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.274933392, 2024-09-16T12:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016378622999999998, 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15521 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302805+0000", 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302805+0000", 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302806+0000", 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0400000000000005e-07 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302809+0000", 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302816+0000", 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0740000000000004e-06 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302821+0000", 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0470000000000001e-06 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318799+0000", 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015978470000000002 2024-09-16T12:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318851+0000", 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1343e-05 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.319183+0000", 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033269199999999999 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:50.705 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:2137 writesame 0xeb8d2c5 thru 0xeb8fdbc data_size 0x32(0x2af8 bytes) 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:2138 read 0x70d49db thru 0x70db087 (0x66ad bytes) 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:2144 read 0x5c8d942 thru 0x5c99707 (0xbdc6 bytes) 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:2145 read 0x7eaaf5f thru 0x7eb3d03 (0x8da5 bytes) 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:2146 read 0x22017b9 thru 0x220ec98 (0xd4e0 bytes) 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:2147 read 0x484c229 thru 0x484f794 (0x356c bytes) 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:2148 read 0xb1cf062 thru 0xb1daef8 (0xbe97 bytes) 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:2149 writesame 0xbd0cc15 thru 0xbd0ea72 data_size 0x1a(0x1e5e bytes) 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:2152 write 0x8063535 thru 0x806919e (0x5c6a bytes) 2024-09-16T12:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout:2153 trunc from 0xeb8fdbd to 0x52e5312 2024-09-16T12:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout:2154 punch from 0x46d539d to 0x46e2c53, (0xd8b6 bytes) 2024-09-16T12:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout:2155 read 0x39c3cc4 thru 0x39d10fd (0xd43a bytes) 2024-09-16T12:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout:2157 punch from 0x50e2167 to 0x50e91db, (0x7074 bytes) 2024-09-16T12:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout:2158 write 0x332e4ef thru 0x3331957 (0x3469 bytes) 2024-09-16T12:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout:2159 trunc from 0x52e5312 to 0xd13bd21 2024-09-16T12:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout:2160 punch from 0x408096a to 0x40869fd, (0x6093 bytes) 2024-09-16T12:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout:2161 write 0x3f1fef7 thru 0x3f20a10 (0xb1a bytes) 2024-09-16T12:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout:2163 write 0x12f3fa8 thru 0x1301f93 (0xdfec bytes) 2024-09-16T12:59:50.709 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:50.699+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout:2164 trunc from 0xd13bd21 to 0x67725b 2024-09-16T12:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout:2165 read 0x58c57b thru 0x593845 (0x72cb bytes) 2024-09-16T12:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout:2166 write 0x446b335 thru 0x447818b (0xce57 bytes) 2024-09-16T12:59:50.807 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:50.802+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:50.808 INFO:teuthology.orchestra.run.smithi042.stdout:2167 write 0x1d9704c thru 0x1d99ae4 (0x2a99 bytes) 2024-09-16T12:59:50.813 INFO:teuthology.orchestra.run.smithi042.stdout:2168 read 0x30a9cd3 thru 0x30b9615 (0xf943 bytes) 2024-09-16T12:59:50.813 INFO:teuthology.orchestra.run.smithi042.stdout:2169 punch from 0x11670bc to 0x1172084, (0xafc8 bytes) 2024-09-16T12:59:50.813 INFO:teuthology.orchestra.run.smithi042.stdout:2171 read 0x1434037 thru 0x1436cb1 (0x2c7b bytes) 2024-09-16T12:59:50.827 INFO:teuthology.orchestra.run.smithi042.stdout:2172 writesame 0x823b992 thru 0x8242901 data_size 0x8(0x6f70 bytes) 2024-09-16T12:59:50.827 INFO:teuthology.orchestra.run.smithi042.stdout:2173 trunc from 0x8242902 to 0x80f339b 2024-09-16T12:59:50.829 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:50.829 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:50.835 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.835 INFO:teuthology.orchestra.run.smithi042.stdout:2175 write 0x5f15f44 thru 0x5f1dfae (0x806b bytes) 2024-09-16T12:59:50.839 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:50.847 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.857 INFO:teuthology.orchestra.run.smithi042.stdout:2177 write 0x9bfe5b7 thru 0x9bfe651 (0x9b bytes) 2024-09-16T12:59:50.861 INFO:teuthology.orchestra.run.smithi042.stdout:2178 read 0x6afc4e6 thru 0x6b04559 (0x8074 bytes) 2024-09-16T12:59:50.862 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.862 INFO:teuthology.orchestra.run.smithi042.stdout:2180 read 0x2e8de08 thru 0x2e8facc (0x1cc5 bytes) 2024-09-16T12:59:50.879 INFO:teuthology.orchestra.run.smithi042.stdout:2181 writesame 0xe4fe48b thru 0xe5052ec data_size 0x47(0x6e62 bytes) 2024-09-16T12:59:50.879 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.879 INFO:teuthology.orchestra.run.smithi042.stdout:2184 read 0xa8749a6 thru 0xa87c9de (0x8039 bytes) 2024-09-16T12:59:50.879 INFO:teuthology.orchestra.run.smithi042.stdout:2185 read 0xba2405f thru 0xba2a383 (0x6325 bytes) 2024-09-16T12:59:50.879 INFO:teuthology.orchestra.run.smithi042.stdout:2186 punch from 0x940a376 to 0x941611d, (0xbda7 bytes) 2024-09-16T12:59:50.887 INFO:teuthology.orchestra.run.smithi042.stdout:2188 writesame 0xe6eb42c thru 0xe6ed81c data_size 0x1(0x23f1 bytes) 2024-09-16T12:59:50.887 INFO:teuthology.orchestra.run.smithi042.stdout:2189 write 0x396055a thru 0x396ffe9 (0xfa90 bytes) 2024-09-16T12:59:50.903 INFO:teuthology.orchestra.run.smithi042.stdout:2190 write 0x56ab451 thru 0x56ae330 (0x2ee0 bytes) 2024-09-16T12:59:50.908 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:50.902+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:50.909 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.910 INFO:teuthology.orchestra.run.smithi042.stdout:2193 punch from 0x4d84eab to 0x4d8ad83, (0x5ed8 bytes) 2024-09-16T12:59:50.910 INFO:teuthology.orchestra.run.smithi042.stdout:2194 punch from 0xc7a1b7b to 0xc7a99cd, (0x7e52 bytes) 2024-09-16T12:59:50.910 INFO:teuthology.orchestra.run.smithi042.stdout:2195 write 0xc6d4dad thru 0xc6e2554 (0xd7a8 bytes) 2024-09-16T12:59:50.926 INFO:teuthology.orchestra.run.smithi042.stdout:2196 trunc from 0xe6ed81d to 0xc2816d3 2024-09-16T12:59:50.940 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:50.940 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:50.941 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:50.941 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:50.941 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:2200 read 0x9bf9295 thru 0x9c0232d (0x9099 bytes) 2024-09-16T12:59:50.950 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:50.955 INFO:teuthology.orchestra.run.smithi042.stdout:2201 writesame 0xe7f50b8 thru 0xe803c6a data_size 0x1(0xebb3 bytes) 2024-09-16T12:59:50.973 INFO:teuthology.orchestra.run.smithi042.stdout:2202 trunc from 0xe803c6b to 0x2db90ff 2024-09-16T12:59:50.996 INFO:teuthology.orchestra.run.smithi042.stdout:2203 read 0x6799b1 thru 0x67a58c (0xbdc bytes) 2024-09-16T12:59:51.009 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:51.002+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:51.023 INFO:teuthology.orchestra.run.smithi042.stdout:2204 write 0xec12475 thru 0xec12eae (0xa3a bytes) 2024-09-16T12:59:51.028 INFO:teuthology.orchestra.run.smithi042.stdout:2206 read 0x19106f4 thru 0x191ae50 (0xa75d bytes) 2024-09-16T12:59:51.028 INFO:teuthology.orchestra.run.smithi042.stdout:2207 read 0xdea3a3d thru 0xdeadc0e (0xa1d2 bytes) 2024-09-16T12:59:51.028 INFO:teuthology.orchestra.run.smithi042.stdout:2208 read 0x1fc9db1 thru 0x1fd1fcf (0x821f bytes) 2024-09-16T12:59:51.028 INFO:teuthology.orchestra.run.smithi042.stdout:2210 write 0x99f5309 thru 0x99f7efd (0x2bf5 bytes) 2024-09-16T12:59:51.034 INFO:teuthology.orchestra.run.smithi042.stdout:2212 write 0x9d78238 thru 0x9d87e76 (0xfc3f bytes) 2024-09-16T12:59:51.045 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T12:59:51.046 INFO:teuthology.orchestra.run.smithi042.stdout:2213 writesame 0x7c2b07c thru 0x7c30163 data_size 0xc(0x50e8 bytes) 2024-09-16T12:59:51.047 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T12:59:51.053 INFO:teuthology.orchestra.run.smithi042.stdout:2214 writesame 0x2773d8d thru 0x2776532 data_size 0x46(0x27a6 bytes) 2024-09-16T12:59:51.056 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:590 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.712777361999997, 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034258392999999998, 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 590 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339271+0000", 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339281+0000", 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6670000000000008e-06 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339307+0000", 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5907000000000001e-05 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339317+0000", 2024-09-16T12:59:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0356999999999999e-05 2024-09-16T12:59:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339412+0000", 2024-09-16T12:59:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4248999999999998e-05 2024-09-16T12:59:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339533+0000", 2024-09-16T12:59:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012184900000000001 2024-09-16T12:59:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.347489+0000", 2024-09-16T12:59:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079559599999999998 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373423+0000", 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025933971 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373435+0000", 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1924e-05 2024-09-16T12:59:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373492+0000", 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6842999999999998e-05 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373528+0000", 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5063 4.13 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22711298~51120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.742632564000001, 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111447293, 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5063 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309416+0000", 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T12:59:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358861+0000", 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049444717999999999 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358893+0000", 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2279e-05 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358902+0000", 2024-09-16T12:59:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4390000000000006e-06 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359062+0000", 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016014100000000001 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359330+0000", 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026734699999999997 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.362566+0000", 2024-09-16T12:59:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032358769999999999 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.417541+0000", 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054975345000000002 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420706+0000", 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031651380000000001 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420790+0000", 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4283999999999995e-05 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420862+0000", 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1138999999999998e-05 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5225 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.804570+0000", 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.247477243999999, 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027003329999999999, 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5225 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T12:59:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T12:59:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804571+0000", 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6309999999999999e-06 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804579+0000", 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804600+0000", 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1114000000000001e-05 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804613+0000", 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3040000000000001e-05 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804713+0000", 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9854999999999999e-05 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804844+0000", 2024-09-16T12:59:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130963 2024-09-16T12:59:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.810955+0000", 2024-09-16T12:59:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061105880000000001 2024-09-16T12:59:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831495+0000", 2024-09-16T12:59:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020540467 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831507+0000", 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831544+0000", 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7413000000000002e-05 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831573+0000", 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8836e-05 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5504 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.468273+0000", 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.583773638, 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020881448, 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5504 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468276+0000", 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8370000000000001e-06 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468290+0000", 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.383e-05 2024-09-16T12:59:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468327+0000", 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468341+0000", 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.331e-05 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468439+0000", 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8746999999999995e-05 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468593+0000", 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015343999999999999 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.472806+0000", 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042129849999999998 2024-09-16T12:59:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488823+0000", 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016017686999999999 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488936+0000", 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112074 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489029+0000", 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3801000000000005e-05 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489155+0000", 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125248 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8352 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.280585+0000", 2024-09-16T12:59:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.771461848, 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014415468000000001, 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8352 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280585+0000", 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280585+0000", 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280587+0000", 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0229999999999999e-06 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280597+0000", 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6139999999999998e-06 2024-09-16T12:59:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280617+0000", 2024-09-16T12:59:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9969000000000001e-05 2024-09-16T12:59:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280626+0000", 2024-09-16T12:59:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.290184+0000", 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009558024 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.290251+0000", 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6210000000000005e-05 2024-09-16T12:59:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.290656+0000", 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040517800000000002 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294871+0000", 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042150529999999999 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294885+0000", 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4154999999999999e-05 2024-09-16T12:59:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294956+0000", 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1101999999999998e-05 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295000+0000", 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4339999999999999e-05 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8353 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.280701+0000", 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.771345676999999, 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015810297000000001, 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8353 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280701+0000", 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280701+0000", 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280702+0000", 2024-09-16T12:59:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5000000000000001e-07 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280708+0000", 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6130000000000003e-06 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280730+0000", 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.211e-05 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.280737+0000", 2024-09-16T12:59:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.323e-06 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.291663+0000", 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010926282000000001 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.291729+0000", 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5908000000000007e-05 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.291994+0000", 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026483799999999999 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296145+0000", 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004150969 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296448+0000", 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030309 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296484+0000", 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5488999999999999e-05 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.296511+0000", 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7725e-05 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8354 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.281367+0000", 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.770679568, 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016884742000000001, 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8354 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281370+0000", 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3159999999999999e-06 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281382+0000", 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281413+0000", 2024-09-16T12:59:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1519000000000001e-05 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281435+0000", 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1812999999999999e-05 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292521+0000", 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011085915999999999 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292628+0000", 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107311 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293108+0000", 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047976600000000001 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297972+0000", 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048640200000000001 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298174+0000", 2024-09-16T12:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020222 2024-09-16T12:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298226+0000", 2024-09-16T12:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2058000000000002e-05 2024-09-16T12:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298252+0000", 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5743000000000001e-05 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8355 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.282597+0000", 2024-09-16T12:59:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.769449669, 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015824709999999999, 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8355 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282597+0000", 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282597+0000", 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282600+0000", 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3870000000000002e-06 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282613+0000", 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3733e-05 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282634+0000", 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0766000000000001e-05 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.282645+0000", 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0783e-05 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293789+0000", 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011144664 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293857+0000", 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7267999999999994e-05 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.294144+0000", 2024-09-16T12:59:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000287025 2024-09-16T12:59:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297979+0000", 2024-09-16T12:59:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003834811 2024-09-16T12:59:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298314+0000", 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000335092 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298371+0000", 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7046999999999998e-05 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298422+0000", 2024-09-16T12:59:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1134000000000002e-05 2024-09-16T12:59:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8357 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.285472+0000", 2024-09-16T12:59:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.766574853, 2024-09-16T12:59:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015109900000000001, 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8357 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285472+0000", 2024-09-16T12:59:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285472+0000", 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285474+0000", 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9039999999999999e-06 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285481+0000", 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3640000000000001e-06 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285495+0000", 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3963e-05 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.285503+0000", 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.712e-06 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295041+0000", 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095385009999999996 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295111+0000", 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0097000000000003e-05 2024-09-16T12:59:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.295515+0000", 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040313399999999998 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297985+0000", 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024699729999999999 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.300526+0000", 2024-09-16T12:59:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025414029999999998 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.300560+0000", 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4e-05 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.300582+0000", 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1849e-05 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8674 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009917+0000", 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.042129568, 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01501186, 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8674 2024-09-16T12:59:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009917+0000", 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009917+0000", 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009918+0000", 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009920+0000", 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.379e-06 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009924+0000", 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5249999999999999e-06 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009928+0000", 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1200000000000002e-06 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.017969+0000", 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080410150000000003 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.018191+0000", 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022191999999999999 2024-09-16T12:59:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.019035+0000", 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084407199999999996 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.022090+0000", 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030553849999999999 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024810+0000", 2024-09-16T12:59:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027204009999999999 2024-09-16T12:59:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024878+0000", 2024-09-16T12:59:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7704000000000001e-05 2024-09-16T12:59:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024929+0000", 2024-09-16T12:59:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0983000000000003e-05 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8675 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009930+0000", 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.042116397999999, 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015359810999999999, 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8675 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009930+0000", 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009930+0000", 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009931+0000", 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6100000000000002e-07 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009934+0000", 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8559999999999998e-06 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009938+0000", 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5369999999999999e-06 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009942+0000", 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8970000000000001e-06 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020310+0000", 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010367833 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.020475+0000", 2024-09-16T12:59:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016531400000000001 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.021011+0000", 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053554600000000003 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025005+0000", 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039943360000000002 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025130+0000", 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125165 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025226+0000", 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5753000000000003e-05 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025290+0000", 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4213000000000003e-05 2024-09-16T12:59:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8676 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.042101893, 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015451705, 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8676 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009945+0000", 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009948+0000", 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6249999999999999e-06 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009952+0000", 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3449999999999999e-06 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009955+0000", 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0510000000000001e-06 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.022130+0000", 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012175002000000001 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.022734+0000", 2024-09-16T12:59:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060358299999999998 2024-09-16T12:59:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023097+0000", 2024-09-16T12:59:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036315300000000001 2024-09-16T12:59:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025115+0000", 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020180720000000001 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025305+0000", 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018960299999999999 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025342+0000", 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7684e-05 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025397+0000", 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4231000000000003e-05 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8677 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009958+0000", 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.042088597999999, 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01642155, 2024-09-16T12:59:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8677 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009959+0000", 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2899999999999999e-07 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009961+0000", 2024-09-16T12:59:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3760000000000002e-06 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009965+0000", 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009968+0000", 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1190000000000001e-06 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023570+0000", 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013601317999999999 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023695+0000", 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012563400000000001 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024191+0000", 2024-09-16T12:59:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000495434 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026285+0000", 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020948360000000001 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026295+0000", 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8889999999999997e-06 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026338+0000", 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2253999999999999e-05 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026380+0000", 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2153000000000001e-05 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8678 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009972+0000", 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.042074346, 2024-09-16T12:59:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020138508999999999, 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8678 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009973+0000", 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3000000000000002e-07 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009975+0000", 2024-09-16T12:59:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4739999999999999e-06 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009980+0000", 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4399999999999998e-06 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009983+0000", 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1209999999999998e-06 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025464+0000", 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015480918 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025536+0000", 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2212000000000006e-05 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025768+0000", 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000232186 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029964+0000", 2024-09-16T12:59:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041957899999999996 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029981+0000", 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6634e-05 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030070+0000", 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9487999999999994e-05 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030111+0000", 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0915999999999999e-05 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8718 4.1b 4:dbb4ad27:::rbd_data.11b55eb65e78.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.086287+0000", 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.965759385, 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028260378999999999, 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8718 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T12:59:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086290+0000", 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1950000000000002e-06 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086300+0000", 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0458999999999999e-05 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086331+0000", 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1117e-05 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086344+0000", 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3191e-05 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086423+0000", 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8288999999999995e-05 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086530+0000", 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010720799999999999 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.091612+0000", 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050817429999999997 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114274+0000", 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022662832000000001 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114386+0000", 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011155899999999999 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114475+0000", 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9043999999999997e-05 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114548+0000", 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2742000000000003e-05 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:15322 4.d 4:b2b80615:::rbd_data.11b5c2c64266.0000000000000120:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.209660+0000", 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.84238722099999996, 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014916707, 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 15322 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.209660+0000", 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.209660+0000", 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.209662+0000", 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3130000000000001e-06 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.209674+0000", 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1773000000000001e-05 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.209709+0000", 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5445999999999999e-05 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.209722+0000", 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.276e-05 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.209851+0000", 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012898 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.210110+0000", 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025908499999999999 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.211967+0000", 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001857023 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.219542+0000", 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075750829999999998 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.224453+0000", 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049110969999999997 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.224526+0000", 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2826999999999999e-05 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.224576+0000", 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0319999999999999e-05 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:15585 4.c 4:3625b175:::rbd_data.11b5c2c64266.00000000000000b4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.380546+0000", 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.67150091899999997, 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014731769, 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 15585 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380546+0000", 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380546+0000", 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380548+0000", 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3630000000000001e-06 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380560+0000", 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1697e-05 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380595+0000", 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5519000000000003e-05 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380609+0000", 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3380000000000001e-05 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380739+0000", 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130094 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380832+0000", 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3277999999999997e-05 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.383608+0000", 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027762469999999999 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.395151+0000", 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011542815999999999 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.395167+0000", 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5862000000000002e-05 2024-09-16T12:59:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.395225+0000", 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8320999999999999e-05 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.395278+0000", 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2191999999999998e-05 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:15586 4.1 4:80691f79:::rbd_data.11b5c2c64266.00000000000000b5:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.380635+0000", 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.67141175099999995, 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015586134, 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 15586 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380635+0000", 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380635+0000", 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380636+0000", 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.06e-06 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380646+0000", 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6229999999999992e-06 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380677+0000", 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1192999999999998e-05 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380684+0000", 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4660000000000002e-06 2024-09-16T12:59:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380807+0000", 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012251699999999999 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.380938+0000", 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013086799999999999 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.383787+0000", 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028493110000000002 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.396108+0000", 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012320612999999999 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.396115+0000", 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9840000000000004e-06 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.396173+0000", 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7921000000000003e-05 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.396221+0000", 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8578000000000001e-05 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:15589 4.13 4:cebf30b6:::rbd_data.11b5c2c64266.00000000000000ba:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.384298+0000", 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.66774898500000002, 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014121053, 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 15589 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384298+0000", 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384298+0000", 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384301+0000", 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7549999999999999e-06 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384315+0000", 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4531e-05 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384385+0000", 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9654999999999996e-05 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384402+0000", 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7073999999999999e-05 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384554+0000", 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000151877 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384827+0000", 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027375099999999997 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.388227+0000", 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033999189999999999 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.398212+0000", 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099848339999999997 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.398283+0000", 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0649999999999996e-05 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.398374+0000", 2024-09-16T12:59:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1479999999999998e-05 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.398419+0000", 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4527000000000002e-05 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:15590 4.9 4:918520a2:::rbd_data.11b5c2c64266.00000000000000bb:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.384464+0000", 2024-09-16T12:59:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.66758244300000003, 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014829811, 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 15590 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384464+0000", 2024-09-16T12:59:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384464+0000", 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384466+0000", 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.418e-06 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384474+0000", 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384492+0000", 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8559e-05 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384503+0000", 2024-09-16T12:59:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0611999999999999e-05 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.384726+0000", 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022308000000000001 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.385044+0000", 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031817899999999998 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.390058+0000", 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050137250000000001 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.399212+0000", 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091539589999999997 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.399223+0000", 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1043000000000001e-05 2024-09-16T12:59:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.399266+0000", 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3386000000000001e-05 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.399294+0000", 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7759e-05 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.181 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:51.182 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-09-16T12:59:50.249190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":532392067,"num_objects":318,"num_object_clones":70,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":48,"num_read":22535,"num_read_kb":93184,"num_write":3872,"num_write_kb":93961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":5991929,"num_keys_recovered":99,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3131,"ondisk_log_size":3131,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2454508,"kb_used_data":13232,"kb_used_omap":13,"kb_used_meta":2441138,"kb_avail":752520212,"statfs":{"total":773094113280,"available":770580697088,"internally_reserved":0,"allocated":13549568,"data_stored":48354755,"data_compressed":778937,"data_compressed_allocated":3547136,"data_compressed_original":42302820,"omap_allocated":13632,"internal_metadata":2499726016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":77787419,"num_objects":48,"num_object_clones":19,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":2,"num_read":3432,"num_read_kb":36276,"num_write":250,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":968932,"num_keys_recovered":58,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.221214"},"pg_stats":[{"pgid":"4.1f","version":"62'81","reported_seq":652,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.041829+0000","last_change":"2024-09-16T12:59:13.789786+0000","last_active":"2024-09-16T12:59:45.041829+0000","last_peered":"2024-09-16T12:59:45.041829+0000","last_clean":"2024-09-16T12:59:45.041829+0000","last_became_active":"2024-09-16T12:59:11.766798+0000","last_became_peered":"2024-09-16T12:59:11.766798+0000","last_unstale":"2024-09-16T12:59:45.041829+0000","last_undegraded":"2024-09-16T12:59:45.041829+0000","last_fullsized":"2024-09-16T12:59:45.041829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:22.588888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":11871837,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":774,"num_read_kb":7701,"num_write":123,"num_write_kb":762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"62'131","reported_seq":434,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.260125+0000","last_change":"2024-09-16T12:59:14.030480+0000","last_active":"2024-09-16T12:59:45.260125+0000","last_peered":"2024-09-16T12:59:45.260125+0000","last_clean":"2024-09-16T12:59:45.260125+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T12:59:45.260125+0000","last_undegraded":"2024-09-16T12:59:45.260125+0000","last_fullsized":"2024-09-16T12:59:45.260125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049602200000000004,"stat_sum":{"num_bytes":26300986,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":651,"num_read_kb":1443,"num_write":151,"num_write_kb":2497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"62'35","reported_seq":138,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.052166+0000","last_change":"2024-09-16T12:59:41.178204+0000","last_active":"2024-09-16T12:59:45.052166+0000","last_peered":"2024-09-16T12:59:45.052166+0000","last_clean":"2024-09-16T12:59:45.052166+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T12:59:45.052166+0000","last_undegraded":"2024-09-16T12:59:45.052166+0000","last_fullsized":"2024-09-16T12:59:45.052166+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012332339999999999,"stat_sum":{"num_bytes":5990303,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":22,"num_read_kb":268,"num_write":35,"num_write_kb":2973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"62'97","reported_seq":419,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.247825+0000","last_change":"2024-09-16T12:59:41.030038+0000","last_active":"2024-09-16T12:59:45.247825+0000","last_peered":"2024-09-16T12:59:45.247825+0000","last_clean":"2024-09-16T12:59:45.247825+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T12:59:45.247825+0000","last_undegraded":"2024-09-16T12:59:45.247825+0000","last_fullsized":"2024-09-16T12:59:45.247825+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015138090000000001,"stat_sum":{"num_bytes":5022260,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":553,"num_read_kb":818,"num_write":103,"num_write_kb":895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"56'55","reported_seq":267,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.324757+0000","last_change":"2024-09-16T12:59:13.779899+0000","last_active":"2024-09-16T12:59:44.324757+0000","last_peered":"2024-09-16T12:59:44.324757+0000","last_clean":"2024-09-16T12:59:44.324757+0000","last_became_active":"2024-09-16T12:59:11.759998+0000","last_became_peered":"2024-09-16T12:59:11.759998+0000","last_unstale":"2024-09-16T12:59:44.324757+0000","last_undegraded":"2024-09-16T12:59:44.324757+0000","last_fullsized":"2024-09-16T12:59:44.324757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:49:22.613940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002573915,"stat_sum":{"num_bytes":15237120,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":458,"num_read_kb":555,"num_write":42,"num_write_kb":1906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"62'68","reported_seq":850,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.261166+0000","last_change":"2024-09-16T12:59:13.779650+0000","last_active":"2024-09-16T12:59:45.261166+0000","last_peered":"2024-09-16T12:59:45.261166+0000","last_clean":"2024-09-16T12:59:45.261166+0000","last_became_active":"2024-09-16T12:59:11.759751+0000","last_became_peered":"2024-09-16T12:59:11.759751+0000","last_unstale":"2024-09-16T12:59:45.261166+0000","last_undegraded":"2024-09-16T12:59:45.261166+0000","last_fullsized":"2024-09-16T12:59:45.261166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:12.049622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002565964,"stat_sum":{"num_bytes":35032444,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":4,"num_read":782,"num_read_kb":7888,"num_write":64,"num_write_kb":3990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"56'32","reported_seq":81,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.022854+0000","last_change":"2024-09-16T12:59:13.770391+0000","last_active":"2024-09-16T12:59:44.022854+0000","last_peered":"2024-09-16T12:59:44.022854+0000","last_clean":"2024-09-16T12:59:44.022854+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T12:59:44.022854+0000","last_undegraded":"2024-09-16T12:59:44.022854+0000","last_fullsized":"2024-09-16T12:59:44.022854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013782849999999999,"stat_sum":{"num_bytes":1900544,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":16,"num_read_kb":62,"num_write":38,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"62'74","reported_seq":249,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.029331+0000","last_change":"2024-09-16T12:59:13.777900+0000","last_active":"2024-09-16T12:59:45.029331+0000","last_peered":"2024-09-16T12:59:45.029331+0000","last_clean":"2024-09-16T12:59:45.029331+0000","last_became_active":"2024-09-16T12:59:11.767308+0000","last_became_peered":"2024-09-16T12:59:11.767308+0000","last_unstale":"2024-09-16T12:59:45.029331+0000","last_undegraded":"2024-09-16T12:59:45.029331+0000","last_fullsized":"2024-09-16T12:59:45.029331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:40.593369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093274999999999999,"stat_sum":{"num_bytes":10117187,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":408,"num_read_kb":1229,"num_write":115,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"62'194","reported_seq":379,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.052542+0000","last_change":"2024-09-16T12:59:13.770695+0000","last_active":"2024-09-16T12:59:45.052542+0000","last_peered":"2024-09-16T12:59:45.052542+0000","last_clean":"2024-09-16T12:59:45.052542+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T12:59:45.052542+0000","last_undegraded":"2024-09-16T12:59:45.052542+0000","last_fullsized":"2024-09-16T12:59:45.052542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001481132,"stat_sum":{"num_bytes":13321952,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":967,"num_read_kb":1206,"num_write":308,"num_write_kb":627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"61'72","reported_seq":906,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.321796+0000","last_change":"2024-09-16T12:59:14.030762+0000","last_active":"2024-09-16T12:59:44.321796+0000","last_peered":"2024-09-16T12:59:44.321796+0000","last_clean":"2024-09-16T12:59:44.321796+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T12:59:44.321796+0000","last_undegraded":"2024-09-16T12:59:44.321796+0000","last_fullsized":"2024-09-16T12:59:44.321796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685533,"stat_sum":{"num_bytes":18897937,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":1334,"num_read_kb":7432,"num_write":64,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"56'108","reported_seq":469,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.323363+0000","last_change":"2024-09-16T12:59:13.796422+0000","last_active":"2024-09-16T12:59:44.323363+0000","last_peered":"2024-09-16T12:59:44.323363+0000","last_clean":"2024-09-16T12:59:44.323363+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T12:59:44.323363+0000","last_undegraded":"2024-09-16T12:59:44.323363+0000","last_fullsized":"2024-09-16T12:59:44.323363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055065799999999999,"stat_sum":{"num_bytes":30186591,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":700,"num_read_kb":1501,"num_write":123,"num_write_kb":848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"62'58","reported_seq":394,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.052493+0000","last_change":"2024-09-16T12:59:41.127844+0000","last_active":"2024-09-16T12:59:45.052493+0000","last_peered":"2024-09-16T12:59:45.052493+0000","last_clean":"2024-09-16T12:59:45.052493+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T12:59:45.052493+0000","last_undegraded":"2024-09-16T12:59:45.052493+0000","last_fullsized":"2024-09-16T12:59:45.052493+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013482349999999999,"stat_sum":{"num_bytes":8757302,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":723,"num_read_kb":2222,"num_write":104,"num_write_kb":4289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"61'154","reported_seq":1014,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.162633+0000","last_change":"2024-09-16T12:59:13.780086+0000","last_active":"2024-09-16T12:59:44.162633+0000","last_peered":"2024-09-16T12:59:44.162633+0000","last_clean":"2024-09-16T12:59:44.162633+0000","last_became_active":"2024-09-16T12:59:11.770883+0000","last_became_peered":"2024-09-16T12:59:11.770883+0000","last_unstale":"2024-09-16T12:59:44.162633+0000","last_undegraded":"2024-09-16T12:59:44.162633+0000","last_fullsized":"2024-09-16T12:59:44.162633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:22.020502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024518000000000001,"stat_sum":{"num_bytes":23032357,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1407,"num_read_kb":9647,"num_write":187,"num_write_kb":33942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'103","reported_seq":465,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T12:59:47.795896+0000","last_change":"2024-09-16T12:59:41.051975+0000","last_active":"2024-09-16T12:59:47.795896+0000","last_peered":"2024-09-16T12:59:47.795896+0000","last_clean":"2024-09-16T12:59:47.795896+0000","last_became_active":"2024-09-16T12:59:40.994472+0000","last_became_peered":"2024-09-16T12:59:40.994472+0000","last_unstale":"2024-09-16T12:59:47.795896+0000","last_undegraded":"2024-09-16T12:59:47.795896+0000","last_fullsized":"2024-09-16T12:59:47.795896+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:54.091219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061900799999999999,"stat_sum":{"num_bytes":20268615,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":824,"num_read_kb":7009,"num_write":137,"num_write_kb":866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":162626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"59'42","reported_seq":120,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.451131+0000","last_change":"2024-09-16T12:59:13.770311+0000","last_active":"2024-09-16T12:59:44.451131+0000","last_peered":"2024-09-16T12:59:44.451131+0000","last_clean":"2024-09-16T12:59:44.451131+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T12:59:44.451131+0000","last_undegraded":"2024-09-16T12:59:44.451131+0000","last_fullsized":"2024-09-16T12:59:44.451131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413508,"stat_sum":{"num_bytes":15655489,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":65,"num_read_kb":839,"num_write":40,"num_write_kb":3483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"56'6","reported_seq":474,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.137021+0000","last_change":"2024-09-16T12:59:41.065829+0000","last_active":"2024-09-16T12:59:44.137021+0000","last_peered":"2024-09-16T12:59:44.137021+0000","last_clean":"2024-09-16T12:59:44.137021+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T12:59:44.137021+0000","last_undegraded":"2024-09-16T12:59:44.137021+0000","last_fullsized":"2024-09-16T12:59:44.137021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047952,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":309,"num_read_kb":4880,"num_write":5,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.024818+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T12:59:44.024818+0000","last_peered":"2024-09-16T12:59:44.024818+0000","last_clean":"2024-09-16T12:59:44.024818+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T12:59:44.024818+0000","last_undegraded":"2024-09-16T12:59:44.024818+0000","last_fullsized":"2024-09-16T12:59:44.024818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"56'99","reported_seq":433,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.100231+0000","last_change":"2024-09-16T12:59:13.777645+0000","last_active":"2024-09-16T12:59:44.100231+0000","last_peered":"2024-09-16T12:59:44.100231+0000","last_clean":"2024-09-16T12:59:44.100231+0000","last_became_active":"2024-09-16T12:59:11.770325+0000","last_became_peered":"2024-09-16T12:59:11.770325+0000","last_unstale":"2024-09-16T12:59:44.100231+0000","last_undegraded":"2024-09-16T12:59:44.100231+0000","last_fullsized":"2024-09-16T12:59:44.100231+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:06:31.319709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069048899999999997,"stat_sum":{"num_bytes":23731034,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":849,"num_read_kb":858,"num_write":89,"num_write_kb":1004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T12:59:47.100365+0000","last_change":"2024-09-16T12:59:40.993864+0000","last_active":"2024-09-16T12:59:47.100365+0000","last_peered":"2024-09-16T12:59:47.100365+0000","last_clean":"2024-09-16T12:59:47.100365+0000","last_became_active":"2024-09-16T12:59:40.992958+0000","last_became_peered":"2024-09-16T12:59:40.992958+0000","last_unstale":"2024-09-16T12:59:47.100365+0000","last_undegraded":"2024-09-16T12:59:47.100365+0000","last_fullsized":"2024-09-16T12:59:47.100365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:44.409493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"62'183","reported_seq":1033,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.051845+0000","last_change":"2024-09-16T12:59:13.769781+0000","last_active":"2024-09-16T12:59:45.051845+0000","last_peered":"2024-09-16T12:59:45.051845+0000","last_clean":"2024-09-16T12:59:45.051845+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T12:59:45.051845+0000","last_undegraded":"2024-09-16T12:59:45.051845+0000","last_fullsized":"2024-09-16T12:59:45.051845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010481990000000001,"stat_sum":{"num_bytes":1191190,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":2413,"num_read_kb":2081,"num_write":195,"num_write_kb":607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T12:59:47.117789+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T12:59:47.117789+0000","last_peered":"2024-09-16T12:59:47.117789+0000","last_clean":"2024-09-16T12:59:47.117789+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T12:59:47.117789+0000","last_undegraded":"2024-09-16T12:59:47.117789+0000","last_fullsized":"2024-09-16T12:59:47.117789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"56'25","reported_seq":204,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.319781+0000","last_change":"2024-09-16T12:59:14.030767+0000","last_active":"2024-09-16T12:59:44.319781+0000","last_peered":"2024-09-16T12:59:44.319781+0000","last_clean":"2024-09-16T12:59:44.319781+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T12:59:44.319781+0000","last_undegraded":"2024-09-16T12:59:44.319781+0000","last_fullsized":"2024-09-16T12:59:44.319781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000670212,"stat_sum":{"num_bytes":2037262,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":142,"num_read_kb":1266,"num_write":30,"num_write_kb":669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T12:59:47.100263+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T12:59:47.100263+0000","last_peered":"2024-09-16T12:59:47.100263+0000","last_clean":"2024-09-16T12:59:47.100263+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T12:59:47.100263+0000","last_undegraded":"2024-09-16T12:59:47.100263+0000","last_fullsized":"2024-09-16T12:59:47.100263+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'46","reported_seq":68,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T12:59:49.238708+0000","last_change":"2024-09-16T12:59:41.165565+0000","last_active":"2024-09-16T12:59:49.238708+0000","last_peered":"2024-09-16T12:59:49.238708+0000","last_clean":"2024-09-16T12:59:49.238708+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T12:59:49.238708+0000","last_undegraded":"2024-09-16T12:59:49.238708+0000","last_fullsized":"2024-09-16T12:59:49.238708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045226000000000001,"stat_sum":{"num_bytes":12321059,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":374,"num_read_kb":760,"num_write":40,"num_write_kb":416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":46,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.051283+0000","last_change":"2024-09-16T12:59:01.688552+0000","last_active":"2024-09-16T12:59:44.051283+0000","last_peered":"2024-09-16T12:59:44.051283+0000","last_clean":"2024-09-16T12:59:44.051283+0000","last_became_active":"2024-09-16T12:59:01.688068+0000","last_became_peered":"2024-09-16T12:59:01.688068+0000","last_unstale":"2024-09-16T12:59:44.051283+0000","last_undegraded":"2024-09-16T12:59:44.051283+0000","last_fullsized":"2024-09-16T12:59:44.051283+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:01.298345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":81,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.214578+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T12:59:44.214578+0000","last_peered":"2024-09-16T12:59:44.214578+0000","last_clean":"2024-09-16T12:59:44.214578+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T12:59:44.214578+0000","last_undegraded":"2024-09-16T12:59:44.214578+0000","last_fullsized":"2024-09-16T12:59:44.214578+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"64'38","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-16T12:59:49.235053+0000","last_change":"2024-09-16T12:59:41.127904+0000","last_active":"2024-09-16T12:59:49.235053+0000","last_peered":"2024-09-16T12:59:49.235053+0000","last_clean":"2024-09-16T12:59:49.235053+0000","last_became_active":"2024-09-16T12:59:40.995175+0000","last_became_peered":"2024-09-16T12:59:40.995175+0000","last_unstale":"2024-09-16T12:59:49.235053+0000","last_undegraded":"2024-09-16T12:59:49.235053+0000","last_fullsized":"2024-09-16T12:59:49.235053+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:41:43.318183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077242,"stat_sum":{"num_bytes":4368462,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":122,"num_read_kb":206,"num_write":39,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":10102,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T12:59:47.100110+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T12:59:47.100110+0000","last_peered":"2024-09-16T12:59:47.100110+0000","last_clean":"2024-09-16T12:59:47.100110+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T12:59:47.100110+0000","last_undegraded":"2024-09-16T12:59:47.100110+0000","last_fullsized":"2024-09-16T12:59:47.100110+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":56,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.021459+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T12:59:44.021459+0000","last_peered":"2024-09-16T12:59:44.021459+0000","last_clean":"2024-09-16T12:59:44.021459+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T12:59:44.021459+0000","last_undegraded":"2024-09-16T12:59:44.021459+0000","last_fullsized":"2024-09-16T12:59:44.021459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"63'107","reported_seq":106,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T12:59:48.293140+0000","last_change":"2024-09-16T12:59:41.112327+0000","last_active":"2024-09-16T12:59:48.293140+0000","last_peered":"2024-09-16T12:59:48.293140+0000","last_clean":"2024-09-16T12:59:48.293140+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T12:59:48.293140+0000","last_undegraded":"2024-09-16T12:59:48.293140+0000","last_fullsized":"2024-09-16T12:59:48.293140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064957100000000005,"stat_sum":{"num_bytes":47263538,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":303,"num_read_kb":8887,"num_write":129,"num_write_kb":5072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.032532+0000","last_change":"2024-09-16T12:59:01.686741+0000","last_active":"2024-09-16T12:59:44.032532+0000","last_peered":"2024-09-16T12:59:44.032532+0000","last_clean":"2024-09-16T12:59:44.032532+0000","last_became_active":"2024-09-16T12:59:01.686267+0000","last_became_peered":"2024-09-16T12:59:01.686267+0000","last_unstale":"2024-09-16T12:59:44.032532+0000","last_undegraded":"2024-09-16T12:59:44.032532+0000","last_fullsized":"2024-09-16T12:59:44.032532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:13:50.209551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T12:59:47.117735+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T12:59:47.117735+0000","last_peered":"2024-09-16T12:59:47.117735+0000","last_clean":"2024-09-16T12:59:47.117735+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T12:59:47.117735+0000","last_undegraded":"2024-09-16T12:59:47.117735+0000","last_fullsized":"2024-09-16T12:59:47.117735+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"56'25","reported_seq":254,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.032795+0000","last_change":"2024-09-16T12:59:41.090757+0000","last_active":"2024-09-16T12:59:44.032795+0000","last_peered":"2024-09-16T12:59:44.032795+0000","last_clean":"2024-09-16T12:59:44.032795+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T12:59:44.032795+0000","last_undegraded":"2024-09-16T12:59:44.032795+0000","last_fullsized":"2024-09-16T12:59:44.032795+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094804399999999999,"stat_sum":{"num_bytes":3291385,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":151,"num_read_kb":1295,"num_write":27,"num_write_kb":2511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"55'1","reported_seq":46,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.022923+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T12:59:44.022923+0000","last_peered":"2024-09-16T12:59:44.022923+0000","last_clean":"2024-09-16T12:59:44.022923+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T12:59:44.022923+0000","last_undegraded":"2024-09-16T12:59:44.022923+0000","last_fullsized":"2024-09-16T12:59:44.022923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:53:17.807916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":54,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.214895+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T12:59:44.214895+0000","last_peered":"2024-09-16T12:59:44.214895+0000","last_clean":"2024-09-16T12:59:44.214895+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T12:59:44.214895+0000","last_undegraded":"2024-09-16T12:59:44.214895+0000","last_fullsized":"2024-09-16T12:59:44.214895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T12:59:47.117796+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T12:59:47.117796+0000","last_peered":"2024-09-16T12:59:47.117796+0000","last_clean":"2024-09-16T12:59:47.117796+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T12:59:47.117796+0000","last_undegraded":"2024-09-16T12:59:47.117796+0000","last_fullsized":"2024-09-16T12:59:47.117796+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"62'45","reported_seq":233,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.041835+0000","last_change":"2024-09-16T12:59:13.789075+0000","last_active":"2024-09-16T12:59:45.041835+0000","last_peered":"2024-09-16T12:59:45.041835+0000","last_clean":"2024-09-16T12:59:45.041835+0000","last_became_active":"2024-09-16T12:59:11.766779+0000","last_became_peered":"2024-09-16T12:59:11.766779+0000","last_unstale":"2024-09-16T12:59:45.041835+0000","last_undegraded":"2024-09-16T12:59:45.041835+0000","last_fullsized":"2024-09-16T12:59:45.041835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:47.829432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046786199999999999,"stat_sum":{"num_bytes":5403326,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":180,"num_read_kb":345,"num_write":38,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"62'200","reported_seq":442,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.153738+0000","last_change":"2024-09-16T12:59:13.796421+0000","last_active":"2024-09-16T12:59:45.153738+0000","last_peered":"2024-09-16T12:59:45.153738+0000","last_clean":"2024-09-16T12:59:45.153738+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T12:59:45.153738+0000","last_undegraded":"2024-09-16T12:59:45.153738+0000","last_fullsized":"2024-09-16T12:59:45.153738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067313300000000002,"stat_sum":{"num_bytes":31786221,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":986,"num_read_kb":2147,"num_write":304,"num_write_kb":1228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"61'76","reported_seq":188,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.316861+0000","last_change":"2024-09-16T12:59:41.059786+0000","last_active":"2024-09-16T12:59:44.316861+0000","last_peered":"2024-09-16T12:59:44.316861+0000","last_clean":"2024-09-16T12:59:44.316861+0000","last_became_active":"2024-09-16T12:59:40.998975+0000","last_became_peered":"2024-09-16T12:59:40.998975+0000","last_unstale":"2024-09-16T12:59:44.316861+0000","last_undegraded":"2024-09-16T12:59:44.316861+0000","last_fullsized":"2024-09-16T12:59:44.316861+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:02.610608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011000280000000001,"stat_sum":{"num_bytes":5144612,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":199,"num_read_kb":188,"num_write":105,"num_write_kb":4490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":16420,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"62'187","reported_seq":471,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.041833+0000","last_change":"2024-09-16T12:59:44.030680+0000","last_active":"2024-09-16T12:59:45.041833+0000","last_peered":"2024-09-16T12:59:45.041833+0000","last_clean":"2024-09-16T12:59:45.041833+0000","last_became_active":"2024-09-16T12:59:44.028122+0000","last_became_peered":"2024-09-16T12:59:44.028122+0000","last_unstale":"2024-09-16T12:59:45.041833+0000","last_undegraded":"2024-09-16T12:59:45.041833+0000","last_fullsized":"2024-09-16T12:59:45.041833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:27:27.517524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078078700000000002,"stat_sum":{"num_bytes":33114064,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1084,"num_read_kb":1311,"num_write":265,"num_write_kb":1178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33178,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"62'45","reported_seq":153,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.162294+0000","last_change":"2024-09-16T12:59:41.151398+0000","last_active":"2024-09-16T12:59:45.162294+0000","last_peered":"2024-09-16T12:59:45.162294+0000","last_clean":"2024-09-16T12:59:45.162294+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T12:59:45.162294+0000","last_undegraded":"2024-09-16T12:59:45.162294+0000","last_fullsized":"2024-09-16T12:59:45.162294+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053245600000000005,"stat_sum":{"num_bytes":4440176,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":77,"num_read_kb":132,"num_write":52,"num_write_kb":2904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"56'80","reported_seq":790,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.324847+0000","last_change":"2024-09-16T12:59:13.780281+0000","last_active":"2024-09-16T12:59:44.324847+0000","last_peered":"2024-09-16T12:59:44.324847+0000","last_clean":"2024-09-16T12:59:44.324847+0000","last_became_active":"2024-09-16T12:59:11.764080+0000","last_became_peered":"2024-09-16T12:59:11.764080+0000","last_unstale":"2024-09-16T12:59:44.324847+0000","last_undegraded":"2024-09-16T12:59:44.324847+0000","last_fullsized":"2024-09-16T12:59:44.324847+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:55:18.557326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020639640000000002,"stat_sum":{"num_bytes":30590596,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":1159,"num_read_kb":7275,"num_write":60,"num_write_kb":570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"62'54","reported_seq":693,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T12:59:45.237872+0000","last_change":"2024-09-16T12:59:13.780488+0000","last_active":"2024-09-16T12:59:45.237872+0000","last_peered":"2024-09-16T12:59:45.237872+0000","last_clean":"2024-09-16T12:59:45.237872+0000","last_became_active":"2024-09-16T12:59:11.764699+0000","last_became_peered":"2024-09-16T12:59:11.764699+0000","last_unstale":"2024-09-16T12:59:45.237872+0000","last_undegraded":"2024-09-16T12:59:45.237872+0000","last_fullsized":"2024-09-16T12:59:45.237872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:12.151943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001758094,"stat_sum":{"num_bytes":31800892,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":664,"num_read_kb":7115,"num_write":44,"num_write_kb":442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"56'438","reported_seq":939,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T12:59:44.451826+0000","last_change":"2024-09-16T12:59:13.780729+0000","last_active":"2024-09-16T12:59:44.451826+0000","last_peered":"2024-09-16T12:59:44.451826+0000","last_clean":"2024-09-16T12:59:44.451826+0000","last_became_active":"2024-09-16T12:59:11.770833+0000","last_became_peered":"2024-09-16T12:59:11.770833+0000","last_unstale":"2024-09-16T12:59:44.451826+0000","last_undegraded":"2024-09-16T12:59:44.451826+0000","last_fullsized":"2024-09-16T12:59:44.451826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:56.784122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637831,"stat_sum":{"num_bytes":34101981,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":2583,"num_read_kb":3412,"num_write":614,"num_write_kb":5280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"63'96","reported_seq":74,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T12:59:47.932768+0000","last_change":"2024-09-16T12:59:41.023359+0000","last_active":"2024-09-16T12:59:47.932768+0000","last_peered":"2024-09-16T12:59:47.932768+0000","last_clean":"2024-09-16T12:59:47.932768+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T12:59:47.932768+0000","last_undegraded":"2024-09-16T12:59:47.932768+0000","last_fullsized":"2024-09-16T12:59:47.932768+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053943399999999999,"stat_sum":{"num_bytes":13331502,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1190,"num_read_kb":1151,"num_write":99,"num_write_kb":5236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":531801680,"num_objects":312,"num_object_clones":70,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":46,"num_read":22469,"num_read_kb":93129,"num_write":3769,"num_write_kb":92331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":5991891,"num_keys_recovered":99,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7753728,"data_stored":46011391,"data_compressed":792572,"data_compressed_allocated":3473408,"data_compressed_original":42401124,"omap_allocated":0,"internal_metadata":0},"log_size":3054,"ondisk_log_size":3054,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307060,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064780,"statfs":{"total":96636764160,"available":96322334720,"internally_reserved":0,"allocated":1417216,"data_stored":3777590,"data_compressed":32210,"data_compressed_allocated":241664,"data_compressed_original":3107715,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303312,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068528,"statfs":{"total":96636764160,"available":96326172672,"internally_reserved":0,"allocated":1830912,"data_stored":8532626,"data_compressed":124212,"data_compressed_allocated":606208,"data_compressed_original":7779934,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307500,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064340,"statfs":{"total":96636764160,"available":96321884160,"internally_reserved":0,"allocated":1859584,"data_stored":6719012,"data_compressed":64226,"data_compressed_allocated":425984,"data_compressed_original":5827459,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307560,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064280,"statfs":{"total":96636764160,"available":96321822720,"internally_reserved":0,"allocated":1921024,"data_stored":7127531,"data_compressed":85889,"data_compressed_allocated":569344,"data_compressed_original":6303744,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306564,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065276,"statfs":{"total":96636764160,"available":96322842624,"internally_reserved":0,"allocated":905216,"data_stored":1124379,"data_compressed":27819,"data_compressed_allocated":61440,"data_compressed_original":692224,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.373},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.81100000000000005}]},{"osd":3,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.4390000000000001}]},{"osd":5,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.62},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.88}]},{"osd":6,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.97699999999999998}]},{"osd":7,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.2969999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306956,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064884,"statfs":{"total":96636764160,"available":96322441216,"internally_reserved":0,"allocated":1306624,"data_stored":4204540,"data_compressed":128645,"data_compressed_allocated":348160,"data_compressed_original":3690496,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]}]},{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307652,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064188,"statfs":{"total":96636764160,"available":96321728512,"internally_reserved":0,"allocated":2027520,"data_stored":7709635,"data_compressed":217961,"data_compressed_allocated":585728,"data_compressed_original":6823936,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":6.6139999999999999,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.92700000000000005}]},{"osd":1,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.6710000000000003,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.621}]},{"osd":2,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":4.1449999999999996,"5min":4.1449999999999996,"15min":4.1449999999999996},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.0979999999999999,"5min":4.0979999999999999,"15min":4.0979999999999999},"last":0.84499999999999997}]},{"osd":3,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":6.3570000000000002,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.03}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.7889999999999997,"5min":4.7889999999999997,"15min":4.7889999999999997},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.173}]},{"osd":5,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":6.282,"5min":6.282,"15min":6.282},"last":1.135},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.611}]},{"osd":6,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":6.3339999999999996,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.21},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.6879999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307904,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063936,"statfs":{"total":96636764160,"available":96321470464,"internally_reserved":0,"allocated":2281472,"data_stored":9159442,"data_compressed":97975,"data_compressed_allocated":708608,"data_compressed_original":8077312,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":4077241,"data_compressed":122983,"data_compressed_allocated":311296,"data_compressed_original":3903488,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1556480,"data_stored":8823031,"data_compressed":97975,"data_compressed_allocated":708608,"data_compressed_original":8077312,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":828065,"data_compressed":27819,"data_compressed_allocated":61440,"data_compressed_original":692224,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1224704,"data_stored":6819504,"data_compressed":85889,"data_compressed_allocated":569344,"data_compressed_original":6303744,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1163264,"data_stored":6444582,"data_compressed":64226,"data_compressed_allocated":425984,"data_compressed_original":5827459,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1114112,"data_stored":8700491,"data_compressed":152847,"data_compressed_allocated":606208,"data_compressed_original":8238686,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":3535654,"data_compressed":32210,"data_compressed_allocated":241664,"data_compressed_original":3107715,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1245184,"data_stored":6782823,"data_compressed":208623,"data_compressed_allocated":548864,"data_compressed_original":6250496,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T12:59:51.185 INFO:tasks.thrashosds.thrasher:Setting 4.4 to [1, 6, 0, 5] 2024-09-16T12:59:51.185 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.4', '1', '6', '0', '5'] 2024-09-16T12:59:51.186 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 4.4 1 6 0 5 2024-09-16T12:59:51.186 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:51.187 INFO:teuthology.orchestra.run.smithi042.stdout:2216 writesame 0xaf5bd6 thru 0xb00c1c data_size 0x1(0xb047 bytes) 2024-09-16T12:59:51.187 INFO:teuthology.orchestra.run.smithi042.stdout:2218 writesame 0x16ff810 thru 0x170e197 data_size 0xbc(0xe988 bytes) 2024-09-16T12:59:51.187 INFO:teuthology.orchestra.run.smithi042.stdout:2220 punch from 0x101a4d1 to 0x1022ddb, (0x890a bytes) 2024-09-16T12:59:51.187 INFO:teuthology.orchestra.run.smithi042.stdout:2223 writesame 0xddd93e7 thru 0xdddc694 data_size 0xd(0x32ae bytes) 2024-09-16T12:59:51.187 INFO:teuthology.orchestra.run.smithi042.stdout:2224 punch from 0x2def16d to 0x2df53b6, (0x6249 bytes) 2024-09-16T12:59:51.187 INFO:teuthology.orchestra.run.smithi042.stdout:2225 trunc from 0xec12eaf to 0xeea068 2024-09-16T12:59:51.187 INFO:teuthology.orchestra.run.smithi042.stdout:2226 trunc from 0xeea068 to 0x6ec68e5 2024-09-16T12:59:51.188 INFO:teuthology.orchestra.run.smithi042.stdout:2227 writesame 0x3702a12 thru 0x3712673 data_size 0x82(0xfc62 bytes) 2024-09-16T12:59:51.188 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:51.175+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:51.189 INFO:teuthology.orchestra.run.smithi042.stdout:2228 read 0x44ee20d thru 0x44f36f4 (0x54e8 bytes) 2024-09-16T12:59:51.189 INFO:teuthology.orchestra.run.smithi042.stdout:2229 read 0x26d6042 thru 0x26e28d7 (0xc896 bytes) 2024-09-16T12:59:51.190 INFO:teuthology.orchestra.run.smithi042.stdout:2230 trunc from 0x6ec68e5 to 0x6ae9b38 2024-09-16T12:59:51.197 INFO:teuthology.orchestra.run.smithi042.stdout:2231 writesame 0x5f6148a thru 0x5f63a7a data_size 0x1(0x25f1 bytes) 2024-09-16T12:59:51.210 INFO:teuthology.orchestra.run.smithi042.stdout:2232 read 0x1ffc0d6 thru 0x1ffd28b (0x11b6 bytes) 2024-09-16T12:59:51.220 INFO:teuthology.orchestra.run.smithi042.stdout:2233 trunc from 0x6ae9b38 to 0x140d00e 2024-09-16T12:59:51.239 INFO:teuthology.orchestra.run.smithi042.stdout:2234 punch from 0x10f43c0 to 0x10faa2e, (0x666e bytes) 2024-09-16T12:59:51.250 INFO:teuthology.orchestra.run.smithi042.stdout:2238 write 0x2caf68e thru 0x2cb934f (0x9cc2 bytes) 2024-09-16T12:59:51.262 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:51.263 INFO:teuthology.orchestra.run.smithi042.stdout:2240 read 0x2a5b51f thru 0x2a5dafe (0x25e0 bytes) 2024-09-16T12:59:51.282 INFO:teuthology.orchestra.run.smithi042.stdout:2242 write 0xa59b8a5 thru 0xa5a9796 (0xdef2 bytes) 2024-09-16T12:59:51.283 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:51.275+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:51.296 INFO:teuthology.orchestra.run.smithi042.stdout:2243 read 0x1de3525 thru 0x1df03ef (0xcecb bytes) 2024-09-16T12:59:51.296 INFO:teuthology.orchestra.run.smithi042.stdout:2246 writesame 0x2faad86 thru 0x2fb47e7 data_size 0x15(0x9a62 bytes) 2024-09-16T12:59:51.305 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:51.305 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:51.305 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:51.306 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:51.307 INFO:teuthology.orchestra.run.smithi042.stdout:2247 write 0x351444 thru 0x35dd2e (0xc8eb bytes) 2024-09-16T12:59:51.317 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:51.337 INFO:teuthology.orchestra.run.smithi042.stdout:2248 punch from 0x92f9518 to 0x93072a5, (0xdd8d bytes) 2024-09-16T12:59:51.338 INFO:teuthology.orchestra.run.smithi042.stdout:2249 read 0x783931d thru 0x784827b (0xef5f bytes) 2024-09-16T12:59:51.338 INFO:teuthology.orchestra.run.smithi042.stdout:2251 read 0x92352c7 thru 0x923d0fa (0x7e34 bytes) 2024-09-16T12:59:51.347 INFO:teuthology.orchestra.run.smithi042.stdout:2252 trunc from 0xa5a9797 to 0x3b9f9ee 2024-09-16T12:59:51.362 INFO:teuthology.orchestra.run.smithi042.stdout:2253 punch from 0x230101b to 0x230ba40, (0xaa25 bytes) 2024-09-16T12:59:51.371 INFO:teuthology.orchestra.run.smithi042.stdout:2254 writesame 0x4a165bc thru 0x4a1c473 data_size 0x1c(0x5eb8 bytes) 2024-09-16T12:59:51.371 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:51.383 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:51.375+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:51.391 INFO:teuthology.orchestra.run.smithi042.stdout:2257 writesame 0xa04c6ba thru 0xa059dd2 data_size 0x5(0xd719 bytes) 2024-09-16T12:59:51.391 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:51.391 INFO:teuthology.orchestra.run.smithi042.stdout:2259 read 0x493e075 thru 0x494741e (0x93aa bytes) 2024-09-16T12:59:51.391 INFO:teuthology.orchestra.run.smithi042.stdout:2260 punch from 0x458df79 to 0x459da14, (0xfa9b bytes) 2024-09-16T12:59:51.405 INFO:teuthology.orchestra.run.smithi042.stdout:2262 writesame 0xc7f3df8 thru 0xc7f9195 data_size 0x16(0x539e bytes) 2024-09-16T12:59:51.405 INFO:teuthology.orchestra.run.smithi042.stdout:2263 read 0x8b8a972 thru 0x8b8b71a (0xda9 bytes) 2024-09-16T12:59:51.405 INFO:teuthology.orchestra.run.smithi042.stdout:2265 trunc from 0xc7f9196 to 0xc0cfc3f 2024-09-16T12:59:51.407 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:51.407 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:51.407 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:51.407 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:51.407 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:51.416 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:51.424 INFO:teuthology.orchestra.run.smithi042.stdout:2266 writesame 0xc5d7928 thru 0xc5e0605 data_size 0x49(0x8cde bytes) 2024-09-16T12:59:51.424 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:51.424 INFO:teuthology.orchestra.run.smithi042.stdout:2268 punch from 0x2500587 to 0x2507791, (0x720a bytes) 2024-09-16T12:59:51.437 INFO:teuthology.orchestra.run.smithi042.stdout:2271 write 0xe4dbdb6 thru 0xe4e1a70 (0x5cbb bytes) 2024-09-16T12:59:51.449 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:51.449 INFO:teuthology.orchestra.run.smithi042.stdout:2275 punch from 0x1be801 to 0x1c7b88, (0x9387 bytes) 2024-09-16T12:59:51.449 INFO:teuthology.orchestra.run.smithi042.stdout:2276 read 0xd9f18b7 thru 0xd9f39e8 (0x2132 bytes) 2024-09-16T12:59:51.449 INFO:teuthology.orchestra.run.smithi042.stdout:2278 write 0x485c107 thru 0x4860da4 (0x4c9e bytes) 2024-09-16T12:59:51.457 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:51.457 INFO:teuthology.orchestra.run.smithi042.stdout:2281 write 0xbd4c1ac thru 0xbd5b142 (0xef97 bytes) 2024-09-16T12:59:51.472 INFO:teuthology.orchestra.run.smithi042.stdout:2284 writesame 0xa5cbe0d thru 0xa5d70ce data_size 0x57(0xb2c2 bytes) 2024-09-16T12:59:51.484 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:51.478+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:51.503 INFO:teuthology.orchestra.run.smithi042.stdout:2287 trunc from 0xe4e1a71 to 0x1880cf9 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8419 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.402892+0000", 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.118355248, 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040293592000000003, 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8419 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T12:59:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402894+0000", 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402907+0000", 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402940+0000", 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3436000000000003e-05 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402953+0000", 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2840000000000001e-05 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403104+0000", 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150712 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403202+0000", 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8403e-05 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.411015+0000", 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078133079999999997 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443018+0000", 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032002348999999999 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443075+0000", 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7703e-05 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443141+0000", 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5705999999999997e-05 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443185+0000", 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4152e-05 2024-09-16T12:59:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:8419 4.7 e54/40 ondisk, result = 0)", 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.431024+0000", 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.090222695, 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012172095000000001, 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431024+0000", 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431024+0000", 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431025+0000", 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3820000000000001e-06 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431032+0000", 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4389999999999997e-06 2024-09-16T12:59:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431054+0000", 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2008e-05 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431064+0000", 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-05 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443061+0000", 2024-09-16T12:59:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011997017 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443069+0000", 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7309999999999993e-06 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443196+0000", 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012701800000000001 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8612 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.847921+0000", 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.673325584000001, 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039491146999999997, 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8612 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847921+0000", 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847921+0000", 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847924+0000", 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0199999999999999e-06 2024-09-16T12:59:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847936+0000", 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1579999999999999e-05 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847968+0000", 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2033000000000003e-05 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847986+0000", 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7839e-05 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.848125+0000", 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013962800000000001 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.848277+0000", 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015126199999999999 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.856740+0000", 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084638479999999995 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887229+0000", 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030488491 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887253+0000", 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4409e-05 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887347+0000", 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3467999999999993e-05 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887412+0000", 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5568999999999995e-05 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:15878 4.15 4:af50264e:::rbd_data.11b5c2c64266.0000000000000129:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.552855+0000", 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.96839141799999995, 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011857385, 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 15878 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552855+0000", 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552855+0000", 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552858+0000", 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2960000000000001e-06 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552869+0000", 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1197e-05 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552902+0000", 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3219000000000002e-05 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552915+0000", 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.257e-05 2024-09-16T12:59:51.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.553032+0000", 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117784 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.553314+0000", 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028161000000000001 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.555090+0000", 2024-09-16T12:59:51.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001776047 2024-09-16T12:59:51.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.563585+0000", 2024-09-16T12:59:51.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084951169999999999 2024-09-16T12:59:51.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.564623+0000", 2024-09-16T12:59:51.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001037638 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.564683+0000", 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9889999999999997e-05 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.564713+0000", 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0017000000000001e-05 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.545 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16272 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754063+0000", 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76718392800000001, 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010178506, 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16272 2024-09-16T12:59:51.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754063+0000", 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754063+0000", 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754063+0000", 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3999999999999997e-07 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754065+0000", 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.576e-06 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754069+0000", 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4129999999999999e-06 2024-09-16T12:59:51.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754075+0000", 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0619999999999997e-06 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764110+0000", 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010034839 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764177+0000", 2024-09-16T12:59:51.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7205000000000005e-05 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764241+0000", 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4071000000000004e-05 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16273 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754081+0000", 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76716608500000005, 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010285459, 2024-09-16T12:59:51.550 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16273 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754081+0000", 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754081+0000", 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754081+0000", 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3000000000000001e-07 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754084+0000", 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.525e-06 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754091+0000", 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7319999999999998e-06 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754096+0000", 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7099999999999998e-06 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764261+0000", 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010164566999999999 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764304+0000", 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3316000000000003e-05 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764366+0000", 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2179000000000002e-05 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16274 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 294912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754101+0000", 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76714589, 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010382317, 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16274 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754101+0000", 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754101+0000", 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754102+0000", 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2099999999999996e-07 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754105+0000", 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9950000000000001e-06 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754113+0000", 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4579999999999993e-06 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754118+0000", 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.843e-06 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764384+0000", 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010266566 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764428+0000", 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3377999999999997e-05 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764483+0000", 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5455999999999999e-05 2024-09-16T12:59:51.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16275 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754123+0000", 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76712400800000002, 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010473434, 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.559 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16275 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754123+0000", 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754123+0000", 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754123+0000", 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7300000000000001e-07 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754139+0000", 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5458999999999999e-05 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754147+0000", 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6109999999999994e-06 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754153+0000", 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6400000000000002e-06 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764501+0000", 2024-09-16T12:59:51.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010347656 2024-09-16T12:59:51.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764543+0000", 2024-09-16T12:59:51.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2088000000000001e-05 2024-09-16T12:59:51.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764596+0000", 2024-09-16T12:59:51.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3507000000000001e-05 2024-09-16T12:59:51.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.564 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.564 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16276 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.564 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754158+0000", 2024-09-16T12:59:51.564 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76708839100000004, 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010575052999999999, 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16276 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754158+0000", 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754158+0000", 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754159+0000", 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.44e-07 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754163+0000", 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9360000000000001e-06 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754171+0000", 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7470000000000005e-06 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754177+0000", 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4749999999999998e-06 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764614+0000", 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010436543 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764655+0000", 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1628999999999997e-05 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764733+0000", 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8178999999999998e-05 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16277 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754182+0000", 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76706451200000003, 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010745922, 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16277 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754182+0000", 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754182+0000", 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754183+0000", 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0900000000000002e-07 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754186+0000", 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3440000000000001e-06 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754194+0000", 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1729999999999995e-06 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754199+0000", 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0010000000000001e-06 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764784+0000", 2024-09-16T12:59:51.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010584748 2024-09-16T12:59:51.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764846+0000", 2024-09-16T12:59:51.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2292000000000006e-05 2024-09-16T12:59:51.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764928+0000", 2024-09-16T12:59:51.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1854999999999995e-05 2024-09-16T12:59:51.573 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.573 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.573 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.573 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16295 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 794624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.573 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754204+0000", 2024-09-16T12:59:51.573 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76704253899999997, 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010835667, 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16295 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754204+0000", 2024-09-16T12:59:51.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754204+0000", 2024-09-16T12:59:51.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754205+0000", 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9999999999999996e-07 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754208+0000", 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.492e-06 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754216+0000", 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9859999999999994e-06 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754221+0000", 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3530000000000002e-06 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764947+0000", 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010725656 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764994+0000", 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256000000000003e-05 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765040+0000", 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6023999999999998e-05 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16296 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754246+0000", 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76700106599999995, 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011016732, 2024-09-16T12:59:51.578 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16296 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754246+0000", 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754246+0000", 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754246+0000", 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6300000000000005e-07 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754249+0000", 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2900000000000001e-06 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754254+0000", 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8270000000000003e-06 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754258+0000", 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765106+0000", 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010848343 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765184+0000", 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8220000000000007e-05 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765262+0000", 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8039000000000003e-05 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16297 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 942080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754263+0000", 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76698423100000002, 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011169742, 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16297 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.583 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754263+0000", 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754263+0000", 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754263+0000", 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4799999999999999e-07 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754265+0000", 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.798e-06 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754269+0000", 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0659999999999997e-06 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754272+0000", 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1370000000000002e-06 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765310+0000", 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011038031 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765372+0000", 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1989999999999994e-05 2024-09-16T12:59:51.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765432+0000", 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0371999999999997e-05 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16298 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754275+0000", 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.766972125, 2024-09-16T12:59:51.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011358536000000001, 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16298 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754275+0000", 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754275+0000", 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754275+0000", 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9999999999999998e-07 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754277+0000", 2024-09-16T12:59:51.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4780000000000001e-06 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754281+0000", 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0820000000000001e-06 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754284+0000", 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9050000000000001e-06 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765506+0000", 2024-09-16T12:59:51.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01122248 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765572+0000", 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5458999999999998e-05 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765633+0000", 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1631999999999995e-05 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16299 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754296+0000", 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76695089400000005, 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011521811, 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16299 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754296+0000", 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754296+0000", 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754296+0000", 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.96e-07 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754298+0000", 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9290000000000001e-06 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754303+0000", 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1420000000000001e-06 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754307+0000", 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1470000000000001e-06 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765675+0000", 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011368417 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765737+0000", 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1586000000000003e-05 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765818+0000", 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1193999999999995e-05 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.595 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16300 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754309+0000", 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76693734700000005, 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011692226999999999, 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16300 2024-09-16T12:59:51.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754309+0000", 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754309+0000", 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754310+0000", 2024-09-16T12:59:51.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2800000000000003e-07 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754311+0000", 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754316+0000", 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1540000000000004e-06 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754318+0000", 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7939999999999998e-06 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765846+0000", 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011527720999999999 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765909+0000", 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3056000000000001e-05 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766002+0000", 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2588000000000002e-05 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16301 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754321+0000", 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76692592999999998, 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011860290000000001, 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16301 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754321+0000", 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754321+0000", 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754321+0000", 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2099999999999998e-07 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754323+0000", 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4139999999999999e-06 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754327+0000", 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9820000000000002e-06 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754329+0000", 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.582e-06 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766038+0000", 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011708435999999999 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766099+0000", 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1699999999999995e-05 2024-09-16T12:59:51.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766181+0000", 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1854999999999995e-05 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16302 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754332+0000", 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76691444399999997, 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012047413999999999, 2024-09-16T12:59:51.604 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16302 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754332+0000", 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754332+0000", 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754333+0000", 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.39e-07 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754334+0000", 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3570000000000001e-06 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754338+0000", 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.233e-06 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754341+0000", 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7240000000000001e-06 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766224+0000", 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011882547 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766291+0000", 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7347000000000004e-05 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766380+0000", 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8867000000000001e-05 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16303 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.608 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754344+0000", 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76690315799999997, 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012220444, 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16303 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754344+0000", 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754344+0000", 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754344+0000", 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6699999999999999e-07 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754345+0000", 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0899999999999999e-06 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754349+0000", 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9790000000000004e-06 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754356+0000", 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8279999999999997e-06 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766418+0000", 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012062226000000001 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766488+0000", 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9438999999999997e-05 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766564+0000", 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6514999999999995e-05 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16304 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754361+0000", 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.76688536100000004, 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012417071999999999, 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16304 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754361+0000", 2024-09-16T12:59:51.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754361+0000", 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754362+0000", 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4999999999999998e-07 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754363+0000", 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.28e-06 2024-09-16T12:59:51.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754368+0000", 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.605e-06 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754371+0000", 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9239999999999999e-06 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766605+0000", 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012234282000000001 2024-09-16T12:59:51.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766701+0000", 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6175999999999995e-05 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766779+0000", 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7454999999999996e-05 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.616 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:51.619 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:51.619 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:51.610+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:51.619 INFO:teuthology.orchestra.run.smithi042.stdout:2288 writesame 0x5fe073d thru 0x5fe17bd data_size 0x19(0x1081 bytes) 2024-09-16T12:59:51.619 INFO:teuthology.orchestra.run.smithi042.stdout:2290 punch from 0x3b9925d to 0x3ba3203, (0x9fa6 bytes) 2024-09-16T12:59:51.619 INFO:teuthology.orchestra.run.smithi042.stdout:2291 trunc from 0x5fe17be to 0x80b3ef4 2024-09-16T12:59:51.619 INFO:teuthology.orchestra.run.smithi042.stdout:2293 write 0x4b0713b thru 0x4b12d1c (0xbbe2 bytes) 2024-09-16T12:59:51.620 INFO:teuthology.orchestra.run.smithi042.stdout:2295 punch from 0x437029f to 0x43709c4, (0x725 bytes) 2024-09-16T12:59:51.620 INFO:teuthology.orchestra.run.smithi042.stdout:2296 write 0xa04a62e thru 0xa04d7dc (0x31af bytes) 2024-09-16T12:59:51.620 INFO:teuthology.orchestra.run.smithi042.stdout:2297 write 0xf6187 thru 0x105e25 (0xfc9f bytes) 2024-09-16T12:59:51.620 INFO:teuthology.orchestra.run.smithi042.stdout:2298 writesame 0xa50b8aa thru 0xa51338a data_size 0x1(0x7ae1 bytes) 2024-09-16T12:59:51.629 INFO:teuthology.orchestra.run.smithi042.stdout:2299 trunc from 0xa51338b to 0x2364679 2024-09-16T12:59:51.673 INFO:teuthology.orchestra.run.smithi042.stdout:2301 write 0x72c00a5 thru 0x72c1b2b (0x1a87 bytes) 2024-09-16T12:59:51.680 INFO:teuthology.orchestra.run.smithi042.stdout:2302 read 0x1ff5141 thru 0x1ffbace (0x698e bytes) 2024-09-16T12:59:51.680 INFO:teuthology.orchestra.run.smithi042.stdout:2304 read 0x4e41985 thru 0x4e482e4 (0x6960 bytes) 2024-09-16T12:59:51.680 INFO:teuthology.orchestra.run.smithi042.stdout:2305 read 0x5107534 thru 0x510aef3 (0x39c0 bytes) 2024-09-16T12:59:51.680 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:51.680 INFO:teuthology.orchestra.run.smithi042.stdout:2307 read 0x1773596 thru 0x1780fec (0xda57 bytes) 2024-09-16T12:59:51.694 INFO:teuthology.orchestra.run.smithi042.stdout:2308 trunc from 0x72c1b2c to 0x8820d1 2024-09-16T12:59:51.717 INFO:teuthology.orchestra.run.smithi042.stdout:2310 read 0x4573c0 thru 0x45b68f (0x42d0 bytes) 2024-09-16T12:59:51.717 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:51.719 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:51.714+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:51.727 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:51.727 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:51.727 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:51.727 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:51.730 INFO:teuthology.orchestra.run.smithi042.stdout:2313 trunc from 0x8820d1 to 0x66ac429 2024-09-16T12:59:51.736 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:51.738 INFO:teuthology.orchestra.run.smithi042.stdout:2314 clone 17 order 20 su 524288 sc 14 2024-09-16T12:59:51.819 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:51.814+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:51.834 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:51.834 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:51.834 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:51.834 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:51.834 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:51.843 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:51.920 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:51.911+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:51.939 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.571348123, 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T12:59:51.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T12:59:51.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T12:59:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5076 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.483630452, 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020443647999999998, 2024-09-16T12:59:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5076 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451913+0000", 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451920+0000", 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5459999999999997e-06 2024-09-16T12:59:51.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451938+0000", 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7581999999999999e-05 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451950+0000", 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2033e-05 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452042+0000", 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2533000000000003e-05 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452251+0000", 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020887 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.456761+0000", 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045099959999999996 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472274+0000", 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015512882 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472281+0000", 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3939999999999998e-06 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472324+0000", 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2389999999999999e-05 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472355+0000", 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1436000000000002e-05 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5513 4.13 e46/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 46'80, mlcod=46'78)", 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.400330213, 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010685616, 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.950 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535213+0000", 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1060000000000001e-06 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535222+0000", 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0159999999999997e-06 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535240+0000", 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7926000000000001e-05 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535247+0000", 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5820000000000004e-06 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535345+0000", 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7705000000000001e-05 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535395+0000", 2024-09-16T12:59:51.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9996e-05 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.545844+0000", 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010448687999999999 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.545897+0000", 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3597000000000002e-05 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5580 4.12 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.242507394, 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034176870999999998, 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5580 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693037+0000", 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0650000000000001e-06 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693045+0000", 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693070+0000", 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5341999999999999e-05 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693081+0000", 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0611000000000001e-05 2024-09-16T12:59:51.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693182+0000", 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100437 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693309+0000", 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012741400000000001 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.703090+0000", 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097814800000000004 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726908+0000", 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023817680000000001 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727023+0000", 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011449400000000001 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727130+0000", 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010704099999999999 2024-09-16T12:59:51.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727211+0000", 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1675000000000007e-05 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10272 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.842352+0000", 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0931898689999997, 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095739899999999992, 2024-09-16T12:59:51.960 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10272 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842352+0000", 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842352+0000", 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842353+0000", 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.46e-06 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842359+0000", 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7069999999999997e-06 2024-09-16T12:59:51.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842394+0000", 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4978999999999998e-05 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842404+0000", 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0083e-05 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.851681+0000", 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092770460000000006 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.851791+0000", 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109757 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.851926+0000", 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013495799999999999 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.965 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10273 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:51.965 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.842419+0000", 2024-09-16T12:59:51.965 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0931230230000004, 2024-09-16T12:59:51.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097372450000000003, 2024-09-16T12:59:51.965 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.965 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10273 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842419+0000", 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842419+0000", 2024-09-16T12:59:51.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842420+0000", 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.226e-06 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842426+0000", 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5940000000000001e-06 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842442+0000", 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6116e-05 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842467+0000", 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5483e-05 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.851972+0000", 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095047629999999994 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852085+0000", 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112716 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852156+0000", 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1347000000000007e-05 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.969 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10274 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 983040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.842497+0000", 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0930444490000006, 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098775080000000001, 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10274 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842497+0000", 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842497+0000", 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842499+0000", 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4899999999999999e-06 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842505+0000", 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7270000000000004e-06 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843160+0000", 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00065513599999999996 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843177+0000", 2024-09-16T12:59:51.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7198999999999999e-05 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852180+0000", 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0090027309999999999 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852275+0000", 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5000999999999994e-05 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852375+0000", 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010022399999999999 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10288 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843357+0000", 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0921844210000007, 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095902299999999999, 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10288 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843357+0000", 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843357+0000", 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843359+0000", 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1939999999999999e-06 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843364+0000", 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6729999999999997e-06 2024-09-16T12:59:51.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843381+0000", 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7036000000000001e-05 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843391+0000", 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2099999999999999e-06 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852759+0000", 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093683729999999993 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852827+0000", 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8201999999999995e-05 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852948+0000", 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120542 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.979 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10289 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 786432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:51.979 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843446+0000", 2024-09-16T12:59:51.979 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0920960120000007, 2024-09-16T12:59:51.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097318130000000006, 2024-09-16T12:59:51.979 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.979 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10289 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843446+0000", 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843446+0000", 2024-09-16T12:59:51.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843447+0000", 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.266e-06 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843453+0000", 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9869999999999996e-06 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843472+0000", 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9219e-05 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843483+0000", 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0706999999999999e-05 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852968+0000", 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094851769999999992 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853049+0000", 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0988000000000005e-05 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853178+0000", 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012846900000000001 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.983 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10290 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843495+0000", 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0920469970000006, 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099464589999999995, 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10290 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843495+0000", 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843495+0000", 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843496+0000", 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1200000000000001e-06 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843501+0000", 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8300000000000003e-06 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843520+0000", 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.95e-05 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843530+0000", 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3659999999999998e-06 2024-09-16T12:59:51.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853216+0000", 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096866920000000002 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853334+0000", 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011804899999999999 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853441+0000", 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106902 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10302 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843540+0000", 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0920022580000008, 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010102442999999999, 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10302 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843540+0000", 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843540+0000", 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843541+0000", 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4399999999999998e-07 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843548+0000", 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1420000000000004e-06 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843564+0000", 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6467e-05 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843583+0000", 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8796000000000001e-05 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853492+0000", 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099093549999999999 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853559+0000", 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6657999999999998e-05 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853642+0000", 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3081e-05 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.992 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10303 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843595+0000", 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0919469559999992, 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010278635, 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10303 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843595+0000", 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843595+0000", 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843596+0000", 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.257e-06 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843602+0000", 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2759999999999997e-06 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843836+0000", 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023349199999999999 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843850+0000", 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.449e-05 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853688+0000", 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098377010000000008 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853776+0000", 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7757999999999996e-05 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853874+0000", 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7660999999999999e-05 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10304 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 589824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843869+0000", 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0916724920000007, 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010178438999999999, 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:51.997 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10304 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843869+0000", 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843869+0000", 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843871+0000", 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7129999999999999e-06 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843879+0000", 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7200000000000006e-06 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843914+0000", 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4983e-05 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843925+0000", 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0773e-05 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853912+0000", 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099872550000000004 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853978+0000", 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5925000000000003e-05 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854048+0000", 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0069999999999998e-05 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10305 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 802816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843937+0000", 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0916051929999995, 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010351196, 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:52.001 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10305 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843937+0000", 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843937+0000", 2024-09-16T12:59:52.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843938+0000", 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1829999999999999e-06 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843943+0000", 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5969999999999999e-06 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:52.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843964+0000", 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0999e-05 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843974+0000", 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.55e-06 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854082+0000", 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010108186999999999 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854182+0000", 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100206 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854288+0000", 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105474 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10306 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844006+0000", 2024-09-16T12:59:52.005 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0915362910000006, 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010519868, 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10306 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844006+0000", 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844006+0000", 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844007+0000", 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7230000000000001e-06 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844013+0000", 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3959999999999997e-06 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844085+0000", 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1913000000000001e-05 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844096+0000", 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1296e-05 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854317+0000", 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010221431 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854419+0000", 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010194199999999999 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854525+0000", 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106167 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.009 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10318 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844137+0000", 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0914043939999996, 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01061718, 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10318 2024-09-16T12:59:52.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844137+0000", 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844137+0000", 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:52.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844139+0000", 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844146+0000", 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7100000000000001e-06 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844178+0000", 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2132999999999999e-05 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844206+0000", 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8685999999999999e-05 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854560+0000", 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010353354 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854650+0000", 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9951999999999995e-05 2024-09-16T12:59:52.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854755+0000", 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104999 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10319 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 393216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844231+0000", 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0913112599999995, 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010766881000000001, 2024-09-16T12:59:52.014 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10319 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844231+0000", 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844231+0000", 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844232+0000", 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.24e-06 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844237+0000", 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5899999999999998e-06 2024-09-16T12:59:52.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844254+0000", 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7042999999999999e-05 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844264+0000", 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2569999999999996e-06 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:52.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854791+0000", 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010527509000000001 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854883+0000", 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2029999999999998e-05 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854997+0000", 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114212 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:52.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10320 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844274+0000", 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0912678269999994, 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010944196999999999, 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10320 2024-09-16T12:59:52.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844274+0000", 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844274+0000", 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844275+0000", 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.127e-06 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844290+0000", 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5352000000000001e-05 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844307+0000", 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6514000000000001e-05 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844317+0000", 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0036e-05 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855027+0000", 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010709556 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855136+0000", 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109378 2024-09-16T12:59:52.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855218+0000", 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2233999999999999e-05 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10321 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844742+0000", 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0908002410000002, 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010659012000000001, 2024-09-16T12:59:52.023 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10321 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844742+0000", 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844742+0000", 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844744+0000", 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.04e-06 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844775+0000", 2024-09-16T12:59:52.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1263000000000002e-05 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844802+0000", 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7302999999999999e-05 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844816+0000", 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3396e-05 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855236+0000", 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01042059 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855295+0000", 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9185999999999999e-05 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855401+0000", 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105234 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.027 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:52.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.028 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10322 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:52.028 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844830+0000", 2024-09-16T12:59:52.028 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0907123389999995, 2024-09-16T12:59:52.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010793654, 2024-09-16T12:59:52.028 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10322 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844830+0000", 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844830+0000", 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844831+0000", 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2899999999999999e-06 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844836+0000", 2024-09-16T12:59:52.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2190000000000003e-06 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844852+0000", 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5687000000000001e-05 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844862+0000", 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0020000000000001e-05 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:52.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855425+0000", 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010563371 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855513+0000", 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8294000000000005e-05 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855623+0000", 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010977299999999999 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.032 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:52.033 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:52.033 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:52.034 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:52.034 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:52.030+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:52.082 INFO:teuthology.orchestra.run.smithi042.stderr:set 4.4 pg_upmap_items mapping to [1->6,0->5] 2024-09-16T12:59:52.117 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:52.120 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T12:59:52.138 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:52.132+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:52.177 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:52.177 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:52.186 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:52.235 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:52.230+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:52.277 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:52.286 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:52.287 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:52.287 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:52.287 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:52.287 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:52.296 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:52.297 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:52.314 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:52.335 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:52.338 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:52.334+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:52.355 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:52.377 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:52.401 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:52.424 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:52.426 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:52.427 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:52.427 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:52.427 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:52.427 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:52.427 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:52.427 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:52.427 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:52.437 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:52.430+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:52.438 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:52.438 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:52.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:52.537 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:52.528+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:52.551 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:52.551 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:52.560 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:52.638 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:52.634+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:52.706 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:52.707 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:52.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:52.739 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:52.732+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:52.817 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:52.817 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:52.827 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:52.840 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:52.834+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:52.923 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:52.923 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:52.933 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:52.941 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:52.934+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:53.030 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:53.031 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:53.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:53.042 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:53.034+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:53.087 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T12:59:53.142 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:53.134+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:53.148 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:53.148 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:53.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:53.243 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:53.238+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:53.255 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:53.255 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:53.264 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:53.344 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:53.338+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:53.366 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:53.366 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:53.375 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:53.445 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:53.442+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:53.471 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:53.471 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:53.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:53.546 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:53.542+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:53.580 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:53.580 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:53.589 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:53.647 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:53.642+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:53.690 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:53.691 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:53.700 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:53.748 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:53.742+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:53.798 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:53.798 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:53.807 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:53.848 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:53.840+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:53.915 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:53.916 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:53.925 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:53.949 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:53.942+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:54.020 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:54.020 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:54.029 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:54.050 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:54.044+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:54.125 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:54.165 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:54.166 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:54.166 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:54.158+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:54.267 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:54.262+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:54.279 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone16 2024-09-16T12:59:54.316 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:54.316 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:54.325 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:54.368 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:54.360+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:54.417 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-16T12:59:54.423 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:54.423 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:54.432 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:54.438 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:54.482 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:54.476+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:54.513 INFO:teuthology.orchestra.run.smithi042.stdout:2315 write 0xa7e3571 thru 0xa7ed239 (0x9cc9 bytes) 2024-09-16T12:59:54.519 INFO:teuthology.orchestra.run.smithi042.stdout:2316 writesame 0xa6d2336 thru 0xa6d677c data_size 0xb(0x4447 bytes) 2024-09-16T12:59:54.535 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:54.535 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:54.535 INFO:teuthology.orchestra.run.smithi042.stdout:2317 write 0xafa248d thru 0xafb06c7 (0xe23b bytes) 2024-09-16T12:59:54.542 INFO:teuthology.orchestra.run.smithi042.stdout:2318 punch from 0x9e8614a to 0x9e8e320, (0x81d6 bytes) 2024-09-16T12:59:54.545 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout:2320 trunc from 0xafb06c8 to 0x1c1e8da 2024-09-16T12:59:54.570 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:54.564+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:54.599 INFO:teuthology.orchestra.run.smithi042.stdout:2321 writesame 0x5048ebc thru 0x50513b7 data_size 0x6(0x84fc bytes) 2024-09-16T12:59:54.599 INFO:teuthology.orchestra.run.smithi042.stdout:2322 read 0x38f3ea6 thru 0x38fe3aa (0xa505 bytes) 2024-09-16T12:59:54.600 INFO:teuthology.orchestra.run.smithi042.stdout:2324 write 0x3fd4afe thru 0x3fe0202 (0xb705 bytes) 2024-09-16T12:59:54.605 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:54.605 INFO:teuthology.orchestra.run.smithi042.stdout:2326 read 0x38eaba thru 0x39a52c (0xba73 bytes) 2024-09-16T12:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout:2328 read 0x1fb3801 thru 0x1fc057c (0xcd7c bytes) 2024-09-16T12:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout:2329 read 0x27c816 thru 0x282c4e (0x6439 bytes) 2024-09-16T12:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout:2334 write 0x46af4c8 thru 0x46b6d59 (0x7892 bytes) 2024-09-16T12:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout:2335 read 0x42e4140 thru 0x42f02d2 (0xc193 bytes) 2024-09-16T12:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout:2342 writesame 0xc29c7f3 thru 0xc29f911 data_size 0x5(0x311f bytes) 2024-09-16T12:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout:2343 write 0x2c5d7d7 thru 0x2c62581 (0x4dab bytes) 2024-09-16T12:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout:2344 read 0xa8e66df thru 0xa8e7758 (0x107a bytes) 2024-09-16T12:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout:2345 write 0x44d29cc thru 0x44ddf09 (0xb53e bytes) 2024-09-16T12:59:54.646 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:54.646 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:54.656 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout:2346 trunc from 0xc29f912 to 0x501c82e 2024-09-16T12:59:54.671 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:54.664+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:54.672 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:54.694 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:54.696 INFO:teuthology.orchestra.run.smithi042.stdout:2347 writesame 0xd4c5793 thru 0xd4c6b29 data_size 0x11(0x1397 bytes) 2024-09-16T12:59:54.696 INFO:teuthology.orchestra.run.smithi042.stdout:2348 write 0xbd2d548 thru 0xbd32f10 (0x59c9 bytes) 2024-09-16T12:59:54.702 INFO:teuthology.orchestra.run.smithi042.stdout:2349 trunc from 0xd4c6b2a to 0xdf315e4 2024-09-16T12:59:54.713 INFO:teuthology.orchestra.run.smithi042.stdout:2350 read 0x3674074 thru 0x367f746 (0xb6d3 bytes) 2024-09-16T12:59:54.713 INFO:teuthology.orchestra.run.smithi042.stdout:2351 read 0xa0dce22 thru 0xa0e4aea (0x7cc9 bytes) 2024-09-16T12:59:54.713 INFO:teuthology.orchestra.run.smithi042.stdout:2352 writesame 0x91abc01 thru 0x91ad33f data_size 0x1(0x173f bytes) 2024-09-16T12:59:54.719 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:54.723 INFO:teuthology.orchestra.run.smithi042.stdout:2353 read 0x45d6aa8 thru 0x45d8203 (0x175c bytes) 2024-09-16T12:59:54.731 INFO:teuthology.orchestra.run.smithi042.stdout:2354 trunc from 0xdf315e4 to 0xa66dd40 2024-09-16T12:59:54.745 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:54.751 INFO:teuthology.orchestra.run.smithi042.stdout:2356 read 0x1afb0b0 thru 0x1b07007 (0xbf58 bytes) 2024-09-16T12:59:54.751 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:54.752 INFO:teuthology.orchestra.run.smithi042.stdout:2365 read 0x5c0c734 thru 0x5c13bd4 (0x74a1 bytes) 2024-09-16T12:59:54.752 INFO:teuthology.orchestra.run.smithi042.stdout:2366 write 0x783d6b7 thru 0x784360c (0x5f56 bytes) 2024-09-16T12:59:54.756 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:54.756 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:54.756 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:54.756 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:54.756 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:54.762 INFO:teuthology.orchestra.run.smithi042.stdout:2367 trunc from 0xa66dd40 to 0xc3b5a3c 2024-09-16T12:59:54.766 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:54.773 INFO:teuthology.orchestra.run.smithi042.stdout:2368 writesame 0x4ddde00 thru 0x4dec027 data_size 0x4(0xe228 bytes) 2024-09-16T12:59:54.773 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:54.764+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:54.787 INFO:teuthology.orchestra.run.smithi042.stdout:2369 read 0x2a05fa6 thru 0x2a14154 (0xe1af bytes) 2024-09-16T12:59:54.787 INFO:teuthology.orchestra.run.smithi042.stdout:2371 read 0xbcf3289 thru 0xbcfe3cb (0xb143 bytes) 2024-09-16T12:59:54.787 INFO:teuthology.orchestra.run.smithi042.stdout:2372 write 0x7370f03 thru 0x7372206 (0x1304 bytes) 2024-09-16T12:59:54.793 INFO:teuthology.orchestra.run.smithi042.stdout:2373 writesame 0x3b85821 thru 0x3b91b82 data_size 0x59(0xc362 bytes) 2024-09-16T12:59:54.800 INFO:teuthology.orchestra.run.smithi042.stdout:2375 write 0xbc6907a thru 0xbc6aa30 (0x19b7 bytes) 2024-09-16T12:59:54.805 INFO:teuthology.orchestra.run.smithi042.stdout:2376 write 0xbe12443 thru 0xbe127ca (0x388 bytes) 2024-09-16T12:59:54.811 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:54.811 INFO:teuthology.orchestra.run.smithi042.stdout:2378 writesame 0x1900b7d thru 0x1902443 data_size 0x1(0x18c7 bytes) 2024-09-16T12:59:54.822 INFO:teuthology.orchestra.run.smithi042.stdout:2385 punch from 0x8182db5 to 0x81886c7, (0x5912 bytes) 2024-09-16T12:59:54.822 INFO:teuthology.orchestra.run.smithi042.stdout:2387 punch from 0x6f3a88 to 0x6fd4f9, (0x9a71 bytes) 2024-09-16T12:59:54.822 INFO:teuthology.orchestra.run.smithi042.stdout:2388 write 0x6ddbd62 thru 0x6dde1e5 (0x2484 bytes) 2024-09-16T12:59:54.828 INFO:teuthology.orchestra.run.smithi042.stdout:2390 write 0x1db305a thru 0x1dbae6c (0x7e13 bytes) 2024-09-16T12:59:54.847 INFO:teuthology.orchestra.run.smithi042.stdout:2394 write 0xc597f84 thru 0xc59f659 (0x76d6 bytes) 2024-09-16T12:59:54.854 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:54.855 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:54.870 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:54.870 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:54.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:54.870 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:54.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.870 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.501806989999999, 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T12:59:54.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5076 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.414089318999999, 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020443647999999998, 2024-09-16T12:59:54.875 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5076 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451913+0000", 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451920+0000", 2024-09-16T12:59:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5459999999999997e-06 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451938+0000", 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7581999999999999e-05 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451950+0000", 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2033e-05 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452042+0000", 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2533000000000003e-05 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452251+0000", 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020887 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.456761+0000", 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045099959999999996 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472274+0000", 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015512882 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472281+0000", 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3939999999999998e-06 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472324+0000", 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2389999999999999e-05 2024-09-16T12:59:54.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472355+0000", 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1436000000000002e-05 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5513 4.13 e46/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 46'80, mlcod=46'78)", 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.330789079999999, 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010685616, 2024-09-16T12:59:54.881 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535212+0000", 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535213+0000", 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1060000000000001e-06 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535222+0000", 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0159999999999997e-06 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535240+0000", 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7926000000000001e-05 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535247+0000", 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5820000000000004e-06 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535345+0000", 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7705000000000001e-05 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.535395+0000", 2024-09-16T12:59:54.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9996e-05 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.545844+0000", 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010448687999999999 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.545897+0000", 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3597000000000002e-05 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5580 4.12 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.172966260999999, 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034176870999999998, 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5580 2024-09-16T12:59:54.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693037+0000", 2024-09-16T12:59:54.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0650000000000001e-06 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693045+0000", 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693070+0000", 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5341999999999999e-05 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693081+0000", 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0611000000000001e-05 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693182+0000", 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100437 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693309+0000", 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012741400000000001 2024-09-16T12:59:54.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.703090+0000", 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097814800000000004 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726908+0000", 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023817680000000001 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727023+0000", 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011449400000000001 2024-09-16T12:59:54.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727130+0000", 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010704099999999999 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727211+0000", 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1675000000000007e-05 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10272 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.842352+0000", 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.023648736, 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095739899999999992, 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10272 2024-09-16T12:59:54.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842352+0000", 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842352+0000", 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842353+0000", 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.46e-06 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842359+0000", 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7069999999999997e-06 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842394+0000", 2024-09-16T12:59:54.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4978999999999998e-05 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842404+0000", 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0083e-05 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.851681+0000", 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092770460000000006 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.851791+0000", 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109757 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.851926+0000", 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013495799999999999 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10273 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.896 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.842419+0000", 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.023581890000001, 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097372450000000003, 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10273 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842419+0000", 2024-09-16T12:59:54.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842419+0000", 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842420+0000", 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.226e-06 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842426+0000", 2024-09-16T12:59:54.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5940000000000001e-06 2024-09-16T12:59:54.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842442+0000", 2024-09-16T12:59:54.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6116e-05 2024-09-16T12:59:54.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842467+0000", 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5483e-05 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.851972+0000", 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095047629999999994 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852085+0000", 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112716 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852156+0000", 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1347000000000007e-05 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10274 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 983040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.901 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.842497+0000", 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.023503315999999, 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098775080000000001, 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10274 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842497+0000", 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842497+0000", 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842499+0000", 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4899999999999999e-06 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.842505+0000", 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7270000000000004e-06 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843160+0000", 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00065513599999999996 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843177+0000", 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7198999999999999e-05 2024-09-16T12:59:54.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852180+0000", 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0090027309999999999 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852275+0000", 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5000999999999994e-05 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852375+0000", 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010022399999999999 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10288 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843357+0000", 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.022643287999999, 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095902299999999999, 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.906 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10288 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843357+0000", 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843357+0000", 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843359+0000", 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1939999999999999e-06 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843364+0000", 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6729999999999997e-06 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843381+0000", 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7036000000000001e-05 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843391+0000", 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2099999999999999e-06 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852759+0000", 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093683729999999993 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852827+0000", 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8201999999999995e-05 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852948+0000", 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120542 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10289 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 786432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843446+0000", 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.022554878999999, 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097318130000000006, 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10289 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843446+0000", 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843446+0000", 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843447+0000", 2024-09-16T12:59:54.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.266e-06 2024-09-16T12:59:54.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843453+0000", 2024-09-16T12:59:54.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9869999999999996e-06 2024-09-16T12:59:54.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843472+0000", 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9219e-05 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843483+0000", 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0706999999999999e-05 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.852968+0000", 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094851769999999992 2024-09-16T12:59:54.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853049+0000", 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0988000000000005e-05 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853178+0000", 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012846900000000001 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10290 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843495+0000", 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.022505863999999, 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099464589999999995, 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10290 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.916 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843495+0000", 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843495+0000", 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843496+0000", 2024-09-16T12:59:54.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1200000000000001e-06 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843501+0000", 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8300000000000003e-06 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843520+0000", 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.95e-05 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843530+0000", 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3659999999999998e-06 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853216+0000", 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096866920000000002 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853334+0000", 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011804899999999999 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853441+0000", 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106902 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10302 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843540+0000", 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.022461125, 2024-09-16T12:59:54.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010102442999999999, 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10302 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843540+0000", 2024-09-16T12:59:54.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843540+0000", 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843541+0000", 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4399999999999998e-07 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843548+0000", 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1420000000000004e-06 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843564+0000", 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6467e-05 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843583+0000", 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8796000000000001e-05 2024-09-16T12:59:54.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853492+0000", 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099093549999999999 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853559+0000", 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6657999999999998e-05 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853642+0000", 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3081e-05 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10303 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843595+0000", 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.022405823, 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010278635, 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.925 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10303 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843595+0000", 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843595+0000", 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843596+0000", 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.257e-06 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843602+0000", 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2759999999999997e-06 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843836+0000", 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023349199999999999 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843850+0000", 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.449e-05 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853688+0000", 2024-09-16T12:59:54.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098377010000000008 2024-09-16T12:59:54.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853776+0000", 2024-09-16T12:59:54.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7757999999999996e-05 2024-09-16T12:59:54.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853874+0000", 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7660999999999999e-05 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10304 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 589824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843869+0000", 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.022131358999999, 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010178438999999999, 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.930 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10304 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843869+0000", 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843869+0000", 2024-09-16T12:59:54.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843871+0000", 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7129999999999999e-06 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843879+0000", 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7200000000000006e-06 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843914+0000", 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4983e-05 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843925+0000", 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0773e-05 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853912+0000", 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099872550000000004 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.853978+0000", 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5925000000000003e-05 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854048+0000", 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0069999999999998e-05 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.934 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10305 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 802816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.843937+0000", 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.02206406, 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010351196, 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.935 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10305 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843937+0000", 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843937+0000", 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843938+0000", 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1829999999999999e-06 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843943+0000", 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5969999999999999e-06 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843964+0000", 2024-09-16T12:59:54.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0999e-05 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.843974+0000", 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.55e-06 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854082+0000", 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010108186999999999 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854182+0000", 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100206 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854288+0000", 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105474 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10306 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.939 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844006+0000", 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.021995157999999, 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010519868, 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10306 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844006+0000", 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844006+0000", 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844007+0000", 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7230000000000001e-06 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844013+0000", 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3959999999999997e-06 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844085+0000", 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1913000000000001e-05 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844096+0000", 2024-09-16T12:59:54.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1296e-05 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854317+0000", 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010221431 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854419+0000", 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010194199999999999 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854525+0000", 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106167 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10318 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844137+0000", 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.021863261, 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01061718, 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.944 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10318 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844137+0000", 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844137+0000", 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844139+0000", 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844146+0000", 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7100000000000001e-06 2024-09-16T12:59:54.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844178+0000", 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2132999999999999e-05 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844206+0000", 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8685999999999999e-05 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854560+0000", 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010353354 2024-09-16T12:59:54.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854650+0000", 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9951999999999995e-05 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854755+0000", 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104999 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10319 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 393216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844231+0000", 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.021770127, 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010766881000000001, 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10319 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.949 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844231+0000", 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844231+0000", 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844232+0000", 2024-09-16T12:59:54.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.24e-06 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844237+0000", 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5899999999999998e-06 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844254+0000", 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7042999999999999e-05 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844264+0000", 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2569999999999996e-06 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854791+0000", 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010527509000000001 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854883+0000", 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2029999999999998e-05 2024-09-16T12:59:54.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.854997+0000", 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114212 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10320 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844274+0000", 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.021726694, 2024-09-16T12:59:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010944196999999999, 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10320 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844274+0000", 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844274+0000", 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844275+0000", 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.127e-06 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844290+0000", 2024-09-16T12:59:54.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5352000000000001e-05 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844307+0000", 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6514000000000001e-05 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844317+0000", 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0036e-05 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855027+0000", 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010709556 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855136+0000", 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109378 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855218+0000", 2024-09-16T12:59:54.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2233999999999999e-05 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10321 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844742+0000", 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.021259108000001, 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010659012000000001, 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10321 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844742+0000", 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844742+0000", 2024-09-16T12:59:54.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844744+0000", 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.04e-06 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844775+0000", 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1263000000000002e-05 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844802+0000", 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7302999999999999e-05 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844816+0000", 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3396e-05 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855236+0000", 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01042059 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855295+0000", 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9185999999999999e-05 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855401+0000", 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105234 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:10322 4.1f 4:fc9e8aeb:::rbd_data.11b55c5d4be2.0000000000000002:f [read 1032192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T12:59:54.962 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:42.844830+0000", 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.021171206, 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010793654, 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 10322 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844830+0000", 2024-09-16T12:59:54.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844830+0000", 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844831+0000", 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2899999999999999e-06 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844836+0000", 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2190000000000003e-06 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844852+0000", 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5687000000000001e-05 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.844862+0000", 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0020000000000001e-05 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855425+0000", 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010563371 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855513+0000", 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8294000000000005e-05 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:42.855623+0000", 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010977299999999999 2024-09-16T12:59:54.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.967 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.967 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.967 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:54.967 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:54.967 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:54.969 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:54.969 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:54.962+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:54.969 INFO:teuthology.orchestra.run.smithi042.stdout:2397 trunc from 0xc59f65a to 0x5c0c31a 2024-09-16T12:59:54.970 INFO:teuthology.orchestra.run.smithi042.stdout:2398 read 0x3495364 thru 0x34a1beb (0xc888 bytes) 2024-09-16T12:59:54.970 INFO:teuthology.orchestra.run.smithi042.stdout:2400 read 0x46555cc thru 0x4660d80 (0xb7b5 bytes) 2024-09-16T12:59:54.970 INFO:teuthology.orchestra.run.smithi042.stdout:2401 write 0x1349cf thru 0x13d74f (0x8d81 bytes) 2024-09-16T12:59:54.970 INFO:teuthology.orchestra.run.smithi042.stdout:2402 write 0x91a949e thru 0x91ab4b3 (0x2016 bytes) 2024-09-16T12:59:54.970 INFO:teuthology.orchestra.run.smithi042.stdout:2403 write 0xbad2da thru 0xbb51d3 (0x7efa bytes) 2024-09-16T12:59:54.970 INFO:teuthology.orchestra.run.smithi042.stdout:2405 write 0xa154c90 thru 0xa15b5e5 (0x6956 bytes) 2024-09-16T12:59:54.970 INFO:teuthology.orchestra.run.smithi042.stdout:2406 write 0x9abb844 thru 0x9ac57c2 (0x9f7f bytes) 2024-09-16T12:59:54.978 INFO:teuthology.orchestra.run.smithi042.stdout:2407 write 0x3dfad8c thru 0x3e00cf5 (0x5f6a bytes) 2024-09-16T12:59:54.985 INFO:teuthology.orchestra.run.smithi042.stdout:2408 write 0x4dde19a thru 0x4ded899 (0xf700 bytes) 2024-09-16T12:59:54.989 INFO:teuthology.orchestra.run.smithi042.stdout:2409 read 0x26fd8fd thru 0x2707282 (0x9986 bytes) 2024-09-16T12:59:54.990 INFO:teuthology.orchestra.run.smithi042.stdout:2411 write 0x18d92ab thru 0x18e3bca (0xa920 bytes) 2024-09-16T12:59:55.019 INFO:teuthology.orchestra.run.smithi042.stdout:2413 write 0xe24822f thru 0xe25185e (0x9630 bytes) 2024-09-16T12:59:55.028 INFO:teuthology.orchestra.run.smithi042.stdout:2416 write 0x92dc0ae thru 0x92e5585 (0x94d8 bytes) 2024-09-16T12:59:55.034 INFO:teuthology.orchestra.run.smithi042.stdout:2417 write 0xa5ab3ff thru 0xa5ac596 (0x1198 bytes) 2024-09-16T12:59:55.041 INFO:teuthology.orchestra.run.smithi042.stdout:2418 punch from 0x752c3b5 to 0x752fd09, (0x3954 bytes) 2024-09-16T12:59:55.042 INFO:teuthology.orchestra.run.smithi042.stdout:2420 punch from 0xb17b5ef to 0xb18788d, (0xc29e bytes) 2024-09-16T12:59:55.042 INFO:teuthology.orchestra.run.smithi042.stdout:2421 read 0x7ecf98e thru 0x7ed9934 (0x9fa7 bytes) 2024-09-16T12:59:55.042 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:55.042 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:55.042 INFO:teuthology.orchestra.run.smithi042.stdout:2424 read 0x165ea8 thru 0x17098f (0xaae8 bytes) 2024-09-16T12:59:55.043 INFO:teuthology.orchestra.run.smithi042.stdout:2425 write 0x4c95e73 thru 0x4c964d6 (0x664 bytes) 2024-09-16T12:59:55.069 INFO:teuthology.orchestra.run.smithi042.stdout:2426 trunc from 0xe25185f to 0x67ea614 2024-09-16T12:59:55.070 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:55.062+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:55.092 INFO:teuthology.orchestra.run.smithi042.stdout:2427 write 0xb2bcf6 thru 0xb379db (0xbce6 bytes) 2024-09-16T12:59:55.099 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T12:59:55.116 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:55.116 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:55.116 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T12:59:55.116 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:55.125 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:55.133 INFO:teuthology.orchestra.run.smithi042.stdout:2429 writesame 0xd72a74f thru 0xd72ab33 data_size 0x1(0x3e5 bytes) 2024-09-16T12:59:55.133 INFO:teuthology.orchestra.run.smithi042.stdout:2430 punch from 0x4e90ce3 to 0x4e97d09, (0x7026 bytes) 2024-09-16T12:59:55.133 INFO:teuthology.orchestra.run.smithi042.stdout:2431 write 0xa43998e thru 0xa4440a4 (0xa717 bytes) 2024-09-16T12:59:55.139 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:55.139 INFO:teuthology.orchestra.run.smithi042.stdout:2434 punch from 0x761f09d to 0x7628867, (0x97ca bytes) 2024-09-16T12:59:55.140 INFO:teuthology.orchestra.run.smithi042.stdout:2436 write 0x930e8a5 thru 0x930f9f8 (0x1154 bytes) 2024-09-16T12:59:55.145 INFO:teuthology.orchestra.run.smithi042.stdout:2437 write 0x45bba66 thru 0x45c0427 (0x49c2 bytes) 2024-09-16T12:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout:2438 write 0x3dfee6f thru 0x3e0a82f (0xb9c1 bytes) 2024-09-16T12:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout:2440 read 0x78555ae thru 0x78611fe (0xbc51 bytes) 2024-09-16T12:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout:2441 writesame 0x1071918 thru 0x10814b7 data_size 0x30(0xfba0 bytes) 2024-09-16T12:59:55.170 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:55.163+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:55.220 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:55.220 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T12:59:55.220 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T12:59:55.220 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T12:59:55.221 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:55.229 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:55.271 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:55.263+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:55.338 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.783941228000003, 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.339 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T12:59:55.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.345 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.959114643999996, 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010809278, 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.346 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374549+0000", 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.037e-06 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374711+0000", 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016226599999999999 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374734+0000", 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2895999999999999e-05 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374746+0000", 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.221e-05 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.380942+0000", 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061959390000000001 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381063+0000", 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121274 2024-09-16T12:59:55.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381255+0000", 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019110500000000001 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384487+0000", 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032327979999999998 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385296+0000", 2024-09-16T12:59:55.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080888499999999996 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385332+0000", 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5774e-05 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385357+0000", 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5094000000000001e-05 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.958903751999998, 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010976025, 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 45 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.352 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374760+0000", 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.13e-06 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374769+0000", 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5699999999999999e-06 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374819+0000", 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9882000000000001e-05 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374828+0000", 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382329+0000", 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075009009999999999 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382403+0000", 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4208999999999994e-05 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382514+0000", 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111276 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385533+0000", 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030190080000000001 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385564+0000", 2024-09-16T12:59:55.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0926000000000002e-05 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385666+0000", 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101791 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385735+0000", 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8541999999999995e-05 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.357 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.958824180999997, 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.358 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T12:59:55.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T12:59:55.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T12:59:55.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.364 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.958776924999995, 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.365 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T12:59:55.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T12:59:55.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T12:59:55.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T12:59:55.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.958573989000001, 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.372 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T12:59:55.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T12:59:55.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T12:59:55.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T12:59:55.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T12:59:55.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T12:59:55.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T12:59:55.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T12:59:55.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T12:59:55.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:55.378 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.958542726999994, 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.958506964999998, 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T12:59:55.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T12:59:55.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.390 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.958312335000002, 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T12:59:55.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T12:59:55.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T12:59:55.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T12:59:55.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T12:59:55.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T12:59:55.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T12:59:55.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.958265490000002, 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T12:59:55.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T12:59:55.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.414205448999994, 2024-09-16T12:59:55.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010623719, 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 69 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919458+0000", 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.474e-06 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919621+0000", 2024-09-16T12:59:55.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016234800000000001 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919641+0000", 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0336999999999999e-05 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919656+0000", 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4554e-05 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924273+0000", 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046169610000000002 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924411+0000", 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138286 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924627+0000", 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021597399999999999 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928794+0000", 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004167268 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929983+0000", 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011884409999999999 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930039+0000", 2024-09-16T12:59:55.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6363000000000002e-05 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930081+0000", 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1712999999999999e-05 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.413989318999995, 2024-09-16T12:59:55.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T12:59:55.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T12:59:55.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T12:59:55.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T12:59:55.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.413935412000001, 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T12:59:55.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T12:59:55.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T12:59:55.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T12:59:55.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T12:59:55.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T12:59:55.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T12:59:55.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.413893107000007, 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T12:59:55.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T12:59:55.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T12:59:55.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T12:59:55.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T12:59:55.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T12:59:55.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T12:59:55.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T12:59:55.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T12:59:55.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T12:59:55.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T12:59:55.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T12:59:55.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.429 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.413734681999998, 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.430 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T12:59:55.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T12:59:55.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T12:59:55.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T12:59:55.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T12:59:55.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.436 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.413685201000007, 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.437 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T12:59:55.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T12:59:55.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.413554630999997, 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T12:59:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T12:59:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T12:59:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T12:59:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T12:59:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.413373117999996, 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.451 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T12:59:55.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T12:59:55.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T12:59:55.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T12:59:55.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T12:59:55.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T12:59:55.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T12:59:55.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.602845450999993, 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T12:59:55.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.602696496999997, 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T12:59:55.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T12:59:55.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T12:59:55.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T12:59:55.471 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:55.473 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:55.466+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:55.473 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:55.574 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:55.570+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:55.581 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:55.581 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:55.581 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:55.581 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:55.591 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:55.675 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:55.670+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:55.687 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:55.687 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:55.687 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:55.687 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:55.687 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:55.696 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:55.776 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:55.770+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:636 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.325735291999997, 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027791489999999999, 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 636 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472202+0000", 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.01e-06 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472218+0000", 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5724000000000001e-05 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472261+0000", 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2858999999999997e-05 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472277+0000", 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6297999999999999e-05 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472397+0000", 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012026199999999999 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472563+0000", 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016614900000000001 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.480681+0000", 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081177869999999996 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499671+0000", 2024-09-16T12:59:55.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018990289 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499756+0000", 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4511000000000005e-05 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499858+0000", 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102495 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499991+0000", 2024-09-16T12:59:55.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132106 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3577 4.1e 4:7a8a5a14:::rbd_data.11b5c378985b.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.697288839999999, 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027798529999999998, 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3577 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100649+0000", 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1769999999999998e-06 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100659+0000", 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0407e-05 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100689+0000", 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0290999999999998e-05 2024-09-16T12:59:55.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100701+0000", 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1960999999999999e-05 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100800+0000", 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8350000000000005e-05 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.101027+0000", 2024-09-16T12:59:55.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000227736 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.111830+0000", 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010802735000000001 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128285+0000", 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016454389999999999 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128338+0000", 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3112000000000002e-05 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128398+0000", 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0136999999999997e-05 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128444+0000", 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6233999999999998e-05 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5066 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.375866727000002, 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019002768999999999, 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5066 2024-09-16T12:59:55.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422068+0000", 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422070+0000", 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-06 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422082+0000", 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2224e-05 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422120+0000", 2024-09-16T12:59:55.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7662e-05 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422133+0000", 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3336000000000001e-05 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422296+0000", 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016319899999999999 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.422539+0000", 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000243281 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.427723+0000", 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051837580000000001 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.440864+0000", 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013140446 2024-09-16T12:59:55.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.440892+0000", 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8628000000000001e-05 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.441002+0000", 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010946999999999999 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.441070+0000", 2024-09-16T12:59:55.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8595000000000003e-05 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5229 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.959712512999999, 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049016400000000002, 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5229 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838225+0000", 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0340000000000001e-06 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838239+0000", 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4004e-05 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838280+0000", 2024-09-16T12:59:55.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0738000000000002e-05 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838294+0000", 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.482e-05 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838415+0000", 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012084 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838570+0000", 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000155162 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.850428+0000", 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011857826 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887135+0000", 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036706938000000001 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887149+0000", 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3421000000000001e-05 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887202+0000", 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3427000000000002e-05 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887238+0000", 2024-09-16T12:59:55.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6189999999999997e-05 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5396 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.575891389999999, 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035598629999999999, 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5396 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222045+0000", 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9580000000000001e-06 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222055+0000", 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9310000000000003e-06 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222082+0000", 2024-09-16T12:59:55.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6882000000000001e-05 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222092+0000", 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8910000000000007e-06 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222166+0000", 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4437999999999994e-05 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222264+0000", 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7748e-05 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.228318+0000", 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060540109999999998 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257411+0000", 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029093622999999999 2024-09-16T12:59:55.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257500+0000", 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8341e-05 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257580+0000", 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257642+0000", 2024-09-16T12:59:55.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1073000000000003e-05 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15002 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:49.884060+0000", 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.9138739830000002, 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012732824, 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15002 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884060+0000", 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884060+0000", 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884061+0000", 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9400000000000001e-07 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884064+0000", 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2320000000000001e-06 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884070+0000", 2024-09-16T12:59:55.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1820000000000003e-06 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884074+0000", 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1690000000000002e-06 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.890874+0000", 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067992959999999998 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.890997+0000", 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012295700000000001 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.892401+0000", 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001404693 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895098+0000", 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026969170000000001 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.896608+0000", 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001509718 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.896746+0000", 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013795999999999999 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.896793+0000", 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7305999999999997e-05 2024-09-16T12:59:55.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15003 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:49.884079+0000", 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.9138555689999999, 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012855256000000001, 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15003 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884079+0000", 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884079+0000", 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884079+0000", 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-07 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884082+0000", 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.853e-06 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884088+0000", 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.998e-06 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884092+0000", 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7850000000000002e-06 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.893068+0000", 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089760819999999998 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.893203+0000", 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013531400000000001 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.893778+0000", 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00057477600000000004 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895117+0000", 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001339462 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.896850+0000", 2024-09-16T12:59:55.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001732728 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.896898+0000", 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8322999999999998e-05 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.896934+0000", 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5611000000000002e-05 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15004 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:49.884099+0000", 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.9138355230000004, 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013399294000000001, 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15004 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884099+0000", 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884099+0000", 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884099+0000", 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-16T12:59:55.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884105+0000", 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3210000000000004e-06 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884110+0000", 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9229999999999999e-06 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.884114+0000", 2024-09-16T12:59:55.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.208e-06 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895185+0000", 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011070986999999999 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895327+0000", 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142244 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.896021+0000", 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000693762 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.897077+0000", 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001056726 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.897259+0000", 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018131199999999999 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.897461+0000", 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000202103 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.897498+0000", 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7264000000000001e-05 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15068 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:49.939524+0000", 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8584105419999997, 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014909446999999999, 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15068 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.939524+0000", 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.939524+0000", 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.939524+0000", 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3999999999999998e-07 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.939526+0000", 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.793e-06 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.939529+0000", 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1829999999999998e-06 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.939531+0000", 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.007e-06 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.950147+0000", 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010616343 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.950257+0000", 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109213 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.950811+0000", 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00055432400000000003 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.951696+0000", 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088485400000000002 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.954284+0000", 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002588576 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.954388+0000", 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103674 2024-09-16T12:59:55.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.954433+0000", 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5240000000000001e-05 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15320 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.204568+0000", 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.5933662440000003, 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014023192, 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15320 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204568+0000", 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204568+0000", 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204568+0000", 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204571+0000", 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.103e-06 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204574+0000", 2024-09-16T12:59:55.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8179999999999997e-06 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204577+0000", 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4590000000000001e-06 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.210775+0000", 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061980439999999998 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.210872+0000", 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7249000000000004e-05 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.211179+0000", 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030661799999999998 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.213184+0000", 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002005115 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218513+0000", 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053287270000000001 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218550+0000", 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7484999999999998e-05 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218591+0000", 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1241e-05 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15321 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.204579+0000", 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.5933550299999997, 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014968192, 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15321 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.864 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204579+0000", 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204579+0000", 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204580+0000", 2024-09-16T12:59:55.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1300000000000001e-07 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204582+0000", 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.046e-06 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204586+0000", 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8600000000000003e-06 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.204588+0000", 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-06 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.211884+0000", 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072962019999999999 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.211984+0000", 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100084 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.212325+0000", 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000340353 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.214039+0000", 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001713865 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.219075+0000", 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050361060000000003 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.219268+0000", 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000193067 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.219547+0000", 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027978600000000002 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.869 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15613 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.408538+0000", 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3893965020000003, 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013239219999999999, 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15613 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.408538+0000", 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.408538+0000", 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.408538+0000", 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0900000000000002e-07 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.408542+0000", 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3400000000000002e-06 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.408548+0000", 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8939999999999998e-06 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.408552+0000", 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6879999999999998e-06 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.419072+0000", 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010519807000000001 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.419194+0000", 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000122069 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.419685+0000", 2024-09-16T12:59:55.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049061799999999996 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.420818+0000", 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011334649999999999 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.421594+0000", 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077582400000000004 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.421733+0000", 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139426 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.421777+0000", 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3579999999999999e-05 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16351 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.790393+0000", 2024-09-16T12:59:55.875 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0075408939999999, 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012830914000000001, 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16351 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.790393+0000", 2024-09-16T12:59:55.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.790393+0000", 2024-09-16T12:59:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.790394+0000", 2024-09-16T12:59:55.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7400000000000003e-07 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.790402+0000", 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4159999999999998e-06 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.790413+0000", 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1878e-05 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.790425+0000", 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1987999999999999e-05 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.799908+0000", 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.009482585 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.800052+0000", 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000144461 2024-09-16T12:59:55.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.800651+0000", 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059847999999999998 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.803010+0000", 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023586280000000002 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.803075+0000", 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5605999999999995e-05 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.803152+0000", 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6559999999999999e-05 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.803224+0000", 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2638000000000005e-05 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16428 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.885346+0000", 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9125883950000002, 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013111065999999999, 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16428 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.882 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.885346+0000", 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.885346+0000", 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.885347+0000", 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6000000000000002e-07 2024-09-16T12:59:55.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.885352+0000", 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.305e-06 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.885369+0000", 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6954999999999999e-05 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.885386+0000", 2024-09-16T12:59:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7085e-05 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.895328+0000", 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0099421430000000005 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.895473+0000", 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014512600000000001 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.895800+0000", 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032618399999999998 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.896638+0000", 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083870400000000001 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.898353+0000", 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017152129999999999 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.898410+0000", 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6412999999999997e-05 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.898457+0000", 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7077999999999999e-05 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.887 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16591 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.078829+0000", 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.719105238, 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012914042000000001, 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16591 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.078829+0000", 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.078829+0000", 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.078829+0000", 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0100000000000001e-07 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.078831+0000", 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.078835+0000", 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.078837+0000", 2024-09-16T12:59:55.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3099999999999999e-06 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.088244+0000", 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0094069189999999997 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.088348+0000", 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103599 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.088830+0000", 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048248799999999999 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.089844+0000", 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001013524 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.091577+0000", 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001733133 2024-09-16T12:59:55.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.091675+0000", 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8487999999999996e-05 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.091743+0000", 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7828000000000002e-05 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.893 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16662 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.169635+0000", 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6282994789999998, 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012741035, 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16662 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169635+0000", 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169635+0000", 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169636+0000", 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9599999999999999e-07 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169640+0000", 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9099999999999996e-06 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169652+0000", 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1206000000000001e-05 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169658+0000", 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6939999999999998e-06 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.176659+0000", 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070009060000000003 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.176775+0000", 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115909 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.177281+0000", 2024-09-16T12:59:55.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00050606200000000005 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182257+0000", 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049757669999999999 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182269+0000", 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2442999999999999e-05 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182332+0000", 2024-09-16T12:59:55.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2104e-05 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182376+0000", 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4338000000000001e-05 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16663 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.169688+0000", 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.628246291, 2024-09-16T12:59:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012862637999999999, 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16663 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169688+0000", 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169688+0000", 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169689+0000", 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5000000000000001e-07 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169695+0000", 2024-09-16T12:59:55.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8440000000000003e-06 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169708+0000", 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2798e-05 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169715+0000", 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2099999999999996e-06 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.178105+0000", 2024-09-16T12:59:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008390158 2024-09-16T12:59:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.178257+0000", 2024-09-16T12:59:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015183500000000001 2024-09-16T12:59:55.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.178583+0000", 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032619600000000001 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182390+0000", 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038070859999999999 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182443+0000", 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3449999999999998e-05 2024-09-16T12:59:55.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182504+0000", 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0834000000000001e-05 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182551+0000", 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6377e-05 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16664 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.169722+0000", 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6282120530000004, 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013171095000000001, 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16664 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169722+0000", 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169722+0000", 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169723+0000", 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0399999999999995e-07 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169728+0000", 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6179999999999997e-06 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169738+0000", 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0256e-05 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169746+0000", 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3080000000000008e-06 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.179129+0000", 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.009382484 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.179217+0000", 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8492999999999995e-05 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.179548+0000", 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033102400000000002 2024-09-16T12:59:55.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182398+0000", 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028501300000000002 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182687+0000", 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028878699999999998 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182811+0000", 2024-09-16T12:59:55.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012438 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182893+0000", 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1910999999999996e-05 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16665 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.169753+0000", 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6281808309999999, 2024-09-16T12:59:55.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013364664, 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16665 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169753+0000", 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169753+0000", 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169754+0000", 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9800000000000003e-07 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169758+0000", 2024-09-16T12:59:55.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1439999999999996e-06 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169769+0000", 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0611999999999999e-05 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169775+0000", 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0820000000000004e-06 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.180096+0000", 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010320858 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.180158+0000", 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2422999999999995e-05 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.180452+0000", 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029363499999999999 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182917+0000", 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024653819999999999 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.182985+0000", 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7486999999999999e-05 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.183059+0000", 2024-09-16T12:59:55.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4225000000000002e-05 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.183118+0000", 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9117999999999999e-05 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16666 4.b 4:d6993850:::rbd_object_map.11b5c2c64266:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.169785+0000", 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6281492149999996, 2024-09-16T12:59:55.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013900569, 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16666 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169785+0000", 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169785+0000", 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169786+0000", 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.92e-07 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169790+0000", 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000004e-06 2024-09-16T12:59:55.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169801+0000", 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0203999999999999e-05 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.169807+0000", 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9880000000000001e-06 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:55.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.181293+0000", 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011486143000000001 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.181376+0000", 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3064000000000003e-05 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.181667+0000", 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000291508 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.183157+0000", 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001489579 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.183589+0000", 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043250999999999999 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.183641+0000", 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1888000000000002e-05 2024-09-16T12:59:55.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:55.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:55.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:55.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.183686+0000", 2024-09-16T12:59:55.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4433e-05 2024-09-16T12:59:55.923 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.923 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.923 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.923 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:55.923 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:55.923 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:55.925 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:55.925 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:55.915+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:56.025 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:56.015+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:56.032 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:56.032 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:56.032 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:56.032 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:56.042 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:56.109 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T12:59:56.111 INFO:teuthology.orchestra.run.smithi042.stdout:2443 writesame 0x23d3301 thru 0x23d7516 data_size 0xb(0x4216 bytes) 2024-09-16T12:59:56.123 INFO:teuthology.orchestra.run.smithi042.stdout:2444 read 0x1be38cb thru 0x1bf2154 (0xe88a bytes) 2024-09-16T12:59:56.123 INFO:teuthology.orchestra.run.smithi042.stdout:2445 write 0x3bd90e4 thru 0x3be64ea (0xd407 bytes) 2024-09-16T12:59:56.126 INFO:teuthology.orchestra.run.smithi042.stdout:2447 read 0xcf8412c thru 0xcf93657 (0xf52c bytes) 2024-09-16T12:59:56.126 INFO:teuthology.orchestra.run.smithi042.stdout:2449 read 0xb1a2227 thru 0xb1ad31b (0xb0f5 bytes) 2024-09-16T12:59:56.127 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:56.122+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:56.138 INFO:teuthology.orchestra.run.smithi042.stdout:2450 trunc from 0xd72ab34 to 0x5a78b71 2024-09-16T12:59:56.166 INFO:teuthology.orchestra.run.smithi042.stdout:2452 trunc from 0x5a78b71 to 0xb02237e 2024-09-16T12:59:56.177 INFO:teuthology.orchestra.run.smithi042.stdout:2453 read 0x56ce9cc thru 0x56d332b (0x4960 bytes) 2024-09-16T12:59:56.177 INFO:teuthology.orchestra.run.smithi042.stdout:2454 write 0xed259d thru 0xed2937 (0x39b bytes) 2024-09-16T12:59:56.183 INFO:teuthology.orchestra.run.smithi042.stdout:2456 read 0x1fb0bd0 thru 0x1fbad7c (0xa1ad bytes) 2024-09-16T12:59:56.184 INFO:teuthology.orchestra.run.smithi042.stdout:2457 punch from 0xc3f52c to 0xc4277b, (0x324f bytes) 2024-09-16T12:59:56.184 INFO:teuthology.orchestra.run.smithi042.stdout:2458 write 0x60431f2 thru 0x6049203 (0x6012 bytes) 2024-09-16T12:59:56.191 INFO:teuthology.orchestra.run.smithi042.stdout:2459 read 0x78f6565 thru 0x78f867b (0x2117 bytes) 2024-09-16T12:59:56.191 INFO:teuthology.orchestra.run.smithi042.stdout:2460 write 0xabd03a2 thru 0xabd1bd2 (0x1831 bytes) 2024-09-16T12:59:56.198 INFO:teuthology.orchestra.run.smithi042.stdout:2461 read 0x390f2b5 thru 0x3918f23 (0x9c6f bytes) 2024-09-16T12:59:56.198 INFO:teuthology.orchestra.run.smithi042.stdout:2462 write 0x8d4b1eb thru 0x8d50680 (0x5496 bytes) 2024-09-16T12:59:56.199 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:56.199 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:56.199 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:56.200 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:56.200 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:56.206 INFO:teuthology.orchestra.run.smithi042.stdout:2464 write 0x382892e thru 0x3835d8e (0xd461 bytes) 2024-09-16T12:59:56.209 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:56.213 INFO:teuthology.orchestra.run.smithi042.stdout:2465 read 0xa30b0f1 thru 0xa30f176 (0x4086 bytes) 2024-09-16T12:59:56.227 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:56.219+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:56.228 INFO:teuthology.orchestra.run.smithi042.stdout:2466 write 0xd3f4d1d thru 0xd4028c8 (0xdbac bytes) 2024-09-16T12:59:56.236 INFO:teuthology.orchestra.run.smithi042.stdout:2467 write 0xa4e2ce9 thru 0xa4e99a5 (0x6cbd bytes) 2024-09-16T12:59:56.242 INFO:teuthology.orchestra.run.smithi042.stdout:2468 read 0x9ba9d88 thru 0x9bb89df (0xec58 bytes) 2024-09-16T12:59:56.264 INFO:teuthology.orchestra.run.smithi042.stdout:2469 trunc from 0xd4028c9 to 0xc7c65 2024-09-16T12:59:56.317 INFO:teuthology.orchestra.run.smithi042.stdout:2470 writesame 0x20b6712 thru 0x20b9b26 data_size 0x1(0x3415 bytes) 2024-09-16T12:59:56.329 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:56.329 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:56.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:56.329 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:56.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.329 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17061 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.329 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309089+0000", 2024-09-16T12:59:56.329 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0136407910000003, 2024-09-16T12:59:56.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017211035999999999, 2024-09-16T12:59:56.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17061 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309089+0000", 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309089+0000", 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309089+0000", 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2399999999999999e-07 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309091+0000", 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8050000000000001e-06 2024-09-16T12:59:56.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309097+0000", 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0440000000000001e-06 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309101+0000", 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6380000000000002e-06 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326190+0000", 2024-09-16T12:59:56.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017088728000000001 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326247+0000", 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7367000000000001e-05 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326300+0000", 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-05 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17062 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309105+0000", 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0136248840000004, 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017328626999999999, 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17062 2024-09-16T12:59:56.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309105+0000", 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309105+0000", 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309105+0000", 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9700000000000002e-07 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309107+0000", 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309113+0000", 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8850000000000004e-06 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309117+0000", 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5939999999999998e-06 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326330+0000", 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017213209 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326381+0000", 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1523999999999997e-05 2024-09-16T12:59:56.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326433+0000", 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2064999999999997e-05 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17063 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309120+0000", 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0136093600000002, 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017433209000000002, 2024-09-16T12:59:56.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17063 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309120+0000", 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309120+0000", 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309121+0000", 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309122+0000", 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7090000000000001e-06 2024-09-16T12:59:56.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309128+0000", 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999997e-06 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309131+0000", 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3459999999999998e-06 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326465+0000", 2024-09-16T12:59:56.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017333818000000001 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326503+0000", 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7848000000000001e-05 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326554+0000", 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0779000000000003e-05 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17064 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309135+0000", 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0135947720000003, 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017544561, 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17064 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309135+0000", 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309135+0000", 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309135+0000", 2024-09-16T12:59:56.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9499999999999998e-07 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309137+0000", 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0150000000000002e-06 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309143+0000", 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9619999999999998e-06 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309147+0000", 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6239999999999999e-06 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326582+0000", 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017434999 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326625+0000", 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3473999999999997e-05 2024-09-16T12:59:56.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326679+0000", 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4092000000000003e-05 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17065 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 393216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309157+0000", 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0135722720000002, 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017644177, 2024-09-16T12:59:56.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17065 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309157+0000", 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309157+0000", 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309158+0000", 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9000000000000002e-07 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309160+0000", 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9e-06 2024-09-16T12:59:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309166+0000", 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5710000000000003e-06 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309170+0000", 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7929999999999999e-06 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326711+0000", 2024-09-16T12:59:56.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017540561 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326749+0000", 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8438e-05 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326802+0000", 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2524000000000001e-05 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17066 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309174+0000", 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.013555384, 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017752123000000002, 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17066 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309174+0000", 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309174+0000", 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309175+0000", 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309177+0000", 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.012e-06 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309183+0000", 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9630000000000003e-06 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309186+0000", 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4180000000000001e-06 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326829+0000", 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017643058999999999 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326874+0000", 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5287000000000002e-05 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326926+0000", 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2030000000000002e-05 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17067 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 557056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309190+0000", 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0135396859999997, 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017933083999999998, 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.356 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17067 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309190+0000", 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309190+0000", 2024-09-16T12:59:56.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309190+0000", 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309192+0000", 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7799999999999999e-06 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309198+0000", 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.677e-06 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309201+0000", 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.411e-06 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326956+0000", 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017754492 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327030+0000", 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4128999999999994e-05 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327123+0000", 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.323e-05 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17068 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 630784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309205+0000", 2024-09-16T12:59:56.360 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0135251070000004, 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018142325000000001, 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17068 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309205+0000", 2024-09-16T12:59:56.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309205+0000", 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309205+0000", 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309213+0000", 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2300000000000008e-06 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309221+0000", 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.52e-06 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309225+0000", 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327152+0000", 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01792728 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327228+0000", 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6197000000000002e-05 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327347+0000", 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118517 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17069 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309229+0000", 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0135007390000004, 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018308606000000002, 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17069 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.365 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309229+0000", 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309229+0000", 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309230+0000", 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2099999999999996e-07 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309233+0000", 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3419999999999999e-06 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309239+0000", 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309243+0000", 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0230000000000003e-06 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327380+0000", 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018136876999999999 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327491+0000", 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011083599999999999 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327538+0000", 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6956999999999998e-05 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17070 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309245+0000", 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0134842409999996, 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018517074000000001, 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.369 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17070 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309245+0000", 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309245+0000", 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309246+0000", 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8000000000000002e-07 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309247+0000", 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.451e-06 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309250+0000", 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0769999999999999e-06 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309252+0000", 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0820000000000001e-06 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327572+0000", 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018319561000000002 2024-09-16T12:59:56.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327667+0000", 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327763+0000", 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5482999999999994e-05 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17071 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309254+0000", 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0134754030000002, 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018675461000000001, 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17071 2024-09-16T12:59:56.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309254+0000", 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309254+0000", 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309255+0000", 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3999999999999998e-07 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309256+0000", 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2100000000000001e-06 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309259+0000", 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0309999999999999e-06 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309261+0000", 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327789+0000", 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018528258999999998 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327840+0000", 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1085000000000003e-05 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327930+0000", 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9683000000000002e-05 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17072 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309263+0000", 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0134670510000001, 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018917891999999999, 2024-09-16T12:59:56.378 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17072 2024-09-16T12:59:56.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.379 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309263+0000", 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309263+0000", 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309263+0000", 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309264+0000", 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.065e-06 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309267+0000", 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8100000000000002e-06 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309269+0000", 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8819999999999999e-06 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327973+0000", 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018703930000000001 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328065+0000", 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2188000000000006e-05 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328181+0000", 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115648 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17073 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309271+0000", 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.013459095, 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019262013000000001, 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.383 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17073 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309271+0000", 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309271+0000", 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309271+0000", 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5400000000000002e-07 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309272+0000", 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.540000000000001e-07 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309275+0000", 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309277+0000", 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.95e-06 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328225+0000", 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018948901000000001 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328318+0000", 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2245999999999997e-05 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328533+0000", 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021494099999999999 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17074 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309278+0000", 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0134513649999999, 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019430942, 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17074 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309278+0000", 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309278+0000", 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309279+0000", 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4900000000000002e-07 2024-09-16T12:59:56.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309280+0000", 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.426e-06 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309283+0000", 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8619999999999999e-06 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309285+0000", 2024-09-16T12:59:56.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328540+0000", 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019254789000000001 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328641+0000", 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010133599999999999 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328709+0000", 2024-09-16T12:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8353000000000001e-05 2024-09-16T12:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17075 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309287+0000", 2024-09-16T12:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0134431140000002, 2024-09-16T12:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019911412, 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17075 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309287+0000", 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309287+0000", 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309287+0000", 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6600000000000003e-07 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309288+0000", 2024-09-16T12:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0449999999999999e-06 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309291+0000", 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8459999999999999e-06 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309293+0000", 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9719999999999999e-06 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328763+0000", 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019470357000000001 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328982+0000", 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021875700000000001 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329198+0000", 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216169 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17076 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309295+0000", 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0134347589999999, 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020043858000000001, 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17076 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.397 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309295+0000", 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309295+0000", 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309295+0000", 2024-09-16T12:59:56.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5199999999999998e-07 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309296+0000", 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0380000000000001e-06 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309299+0000", 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.926e-06 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309301+0000", 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8929999999999999e-06 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329222+0000", 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019920617000000002 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329276+0000", 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4743000000000001e-05 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329339+0000", 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2389000000000002e-05 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17077 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309303+0000", 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0134266680000001, 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020302292999999999, 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17077 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309303+0000", 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309303+0000", 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309303+0000", 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5600000000000002e-07 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309304+0000", 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6899999999999996e-07 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309307+0000", 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0089999999999999e-06 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309309+0000", 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0990000000000001e-06 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329377+0000", 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020068004 2024-09-16T12:59:56.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329477+0000", 2024-09-16T12:59:56.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100134 2024-09-16T12:59:56.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329605+0000", 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012782200000000001 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17078 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309311+0000", 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0134183989999999, 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020446052999999999, 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17078 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309311+0000", 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309311+0000", 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309312+0000", 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4200000000000002e-07 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309313+0000", 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105e-06 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309316+0000", 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8679999999999999e-06 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309317+0000", 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8920000000000001e-06 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329639+0000", 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020322030000000001 2024-09-16T12:59:56.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329706+0000", 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6291000000000006e-05 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329757+0000", 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1625000000000001e-05 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17079 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 663552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309319+0000", 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0134103919999999, 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020674145000000001, 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17079 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309319+0000", 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309319+0000", 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309320+0000", 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3699999999999999e-07 2024-09-16T12:59:56.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309320+0000", 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5099999999999998e-07 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309323+0000", 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8270000000000002e-06 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309325+0000", 2024-09-16T12:59:56.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8899999999999999e-06 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329805+0000", 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020479430999999999 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329901+0000", 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6507000000000003e-05 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329993+0000", 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2301999999999998e-05 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17080 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309329+0000", 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0134010819999997, 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020787672, 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17080 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309329+0000", 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309329+0000", 2024-09-16T12:59:56.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309329+0000", 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3999999999999998e-07 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309330+0000", 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2600000000000001e-07 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309333+0000", 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0410000000000002e-06 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309335+0000", 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0169999999999999e-06 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.330016+0000", 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020681102999999999 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.330070+0000", 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4246999999999997e-05 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.330116+0000", 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6097999999999998e-05 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.419 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:56.420 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:56.322+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:56.421 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:56.422 INFO:teuthology.orchestra.run.smithi042.stdout:2471 write 0x94e375f thru 0x94e9d2d (0x65cf bytes) 2024-09-16T12:59:56.422 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2473 read 0x2990313 thru 0x29924ac (0x219a bytes) 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2476 writesame 0x938a9df thru 0x9399ad7 data_size 0x1(0xf0f9 bytes) 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2478 punch from 0x8296033 to 0x829b068, (0x5035 bytes) 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2480 write 0xc53304 thru 0xc5fe31 (0xcb2e bytes) 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2481 punch from 0x8441d20 to 0x844e734, (0xca14 bytes) 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2483 write 0x680941b thru 0x681083f (0x7425 bytes) 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2484 read 0x8c91993 thru 0x8c9e673 (0xcce1 bytes) 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2486 write 0x4f6f82a thru 0x4f7b3e9 (0xbbc0 bytes) 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2487 read 0x4f51592 thru 0x4f56a03 (0x5472 bytes) 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2488 read 0x79a0410 thru 0x79ad0b7 (0xcca8 bytes) 2024-09-16T12:59:56.423 INFO:teuthology.orchestra.run.smithi042.stdout:2491 read 0x2f15212 thru 0x2f215e8 (0xc3d7 bytes) 2024-09-16T12:59:56.429 INFO:teuthology.orchestra.run.smithi042.stdout:2492 write 0xb17e234 thru 0xb182074 (0x3e41 bytes) 2024-09-16T12:59:56.430 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:56.422+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:56.457 INFO:teuthology.orchestra.run.smithi042.stdout:2496 trunc from 0xb182075 to 0x374ba3a 2024-09-16T12:59:56.479 INFO:teuthology.orchestra.run.smithi042.stdout:2497 write 0x17c50db thru 0x17cc38e (0x72b4 bytes) 2024-09-16T12:59:56.485 INFO:teuthology.orchestra.run.smithi042.stdout:2498 punch from 0x10ec8b6 to 0x10f6fd9, (0xa723 bytes) 2024-09-16T12:59:56.499 INFO:teuthology.orchestra.run.smithi042.stdout:2499 write 0x4df410b thru 0x4e03f0f (0xfe05 bytes) 2024-09-16T12:59:56.508 INFO:teuthology.orchestra.run.smithi042.stdout:2500 read 0xbdbafc thru 0xbe8520 (0xca25 bytes) 2024-09-16T12:59:56.508 INFO:teuthology.orchestra.run.smithi042.stdout:2503 punch from 0x857088 to 0x8625ab, (0xb523 bytes) 2024-09-16T12:59:56.530 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:56.523+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:56.533 INFO:teuthology.orchestra.run.smithi042.stdout:2504 write 0xc50cdd1 thru 0xc51c887 (0xfab7 bytes) 2024-09-16T12:59:56.542 INFO:teuthology.orchestra.run.smithi042.stdout:2505 read 0xa374a06 thru 0xa3821c9 (0xd7c4 bytes) 2024-09-16T12:59:56.543 INFO:teuthology.orchestra.run.smithi042.stdout:2506 trunc from 0xc51c888 to 0xd2e4e17 2024-09-16T12:59:56.550 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:56.550 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:56.550 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T12:59:56.550 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:56.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:56.570 INFO:teuthology.orchestra.run.smithi042.stdout:2507 write 0xdd4af1a thru 0xdd537ae (0x8895 bytes) 2024-09-16T12:59:56.578 INFO:teuthology.orchestra.run.smithi042.stdout:2508 write 0x5897de6 thru 0x589b3bc (0x35d7 bytes) 2024-09-16T12:59:56.585 INFO:teuthology.orchestra.run.smithi042.stdout:2509 read 0x286d54d thru 0x2875e82 (0x8936 bytes) 2024-09-16T12:59:56.585 INFO:teuthology.orchestra.run.smithi042.stdout:2510 read 0x7ca8bba thru 0x7ca9528 (0x96f bytes) 2024-09-16T12:59:56.585 INFO:teuthology.orchestra.run.smithi042.stdout:2512 writesame 0x2ae005b thru 0x2aea897 data_size 0x1(0xa83d bytes) 2024-09-16T12:59:56.621 INFO:teuthology.orchestra.run.smithi042.stdout:2513 write 0xab82f70 thru 0xab8d48b (0xa51c bytes) 2024-09-16T12:59:56.627 INFO:teuthology.orchestra.run.smithi042.stdout:2514 clone 18 order 22 su 131072 sc 14 2024-09-16T12:59:56.632 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:56.626+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:56.667 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:56.668 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:56.668 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:56.668 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:56.668 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:56.677 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:56.733 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:56.726+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:56.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T12:59:56.820 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.035091569000002, 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003284908, 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 10 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779655+0000", 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779657+0000", 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1239999999999999e-06 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779667+0000", 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0487e-05 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779683+0000", 2024-09-16T12:59:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6146000000000001e-05 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779695+0000", 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1347e-05 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779769+0000", 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.449e-05 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.779918+0000", 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000148462 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.780088+0000", 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017066600000000001 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.781019+0000", 2024-09-16T12:59:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093024399999999999 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782840+0000", 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018211379999999999 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782899+0000", 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8777000000000003e-05 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:13.782940+0000", 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1026999999999999e-05 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:681 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [set-alloc-hint object_size 1048576 write_size 1048576,write 119408~11664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.206185499, 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035485299999999998, 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 681 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608564+0000", 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.146e-06 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608592+0000", 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8585000000000002e-05 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608635+0000", 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629000000000001e-05 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608649+0000", 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4270000000000001e-05 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608797+0000", 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014800499999999999 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609162+0000", 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036455100000000001 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609411+0000", 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024895400000000002 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611008+0000", 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015968860000000001 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611948+0000", 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094050999999999998 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612013+0000", 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5185999999999996e-05 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612109+0000", 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5808000000000002e-05 2024-09-16T12:59:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:810 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 443751~14451] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.918101968999999, 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036011279999999999, 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 810 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896648+0000", 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6109999999999998e-06 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896701+0000", 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3168999999999999e-05 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896743+0000", 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1445e-05 2024-09-16T12:59:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896758+0000", 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5591e-05 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896878+0000", 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120256 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897013+0000", 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134327 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897221+0000", 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020804800000000001 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.899109+0000", 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018886899999999999 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900084+0000", 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097495899999999996 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900178+0000", 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3264999999999996e-05 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900245+0000", 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7767000000000004e-05 2024-09-16T12:59:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:860 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [set-alloc-hint object_size 1048576 write_size 1048576,write 997016~9314] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.816470840999997, 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037930170000000001, 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 860 2024-09-16T12:59:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998280+0000", 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0969999999999999e-06 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998307+0000", 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7804999999999999e-05 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998349+0000", 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1838000000000002e-05 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998364+0000", 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5194000000000001e-05 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998511+0000", 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014693900000000001 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998912+0000", 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000401066 2024-09-16T12:59:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.999376+0000", 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000463938 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001244+0000", 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018672389999999999 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001920+0000", 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067617100000000004 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002008+0000", 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8342000000000002e-05 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002068+0000", 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0387999999999998e-05 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:995 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.507589748000001, 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040928859999999996, 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 995 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307159+0000", 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9739999999999998e-06 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307174+0000", 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4134000000000001e-05 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307211+0000", 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7385000000000002e-05 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307230+0000", 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8961e-05 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307461+0000", 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230721 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307710+0000", 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024951200000000002 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308016+0000", 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030602199999999998 2024-09-16T12:59:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309601+0000", 2024-09-16T12:59:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015845200000000001 2024-09-16T12:59:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311094+0000", 2024-09-16T12:59:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00149321 2024-09-16T12:59:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311180+0000", 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6142000000000003e-05 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311249+0000", 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9305000000000001e-05 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:4044 4.1c 4:3cacb79f:::rbd_object_map.11b5d5152bdd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.340292198999997, 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002965217, 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 4044 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474454+0000", 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474456+0000", 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.818e-06 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474465+0000", 2024-09-16T12:59:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5459999999999997e-06 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474490+0000", 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5058e-05 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474502+0000", 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1110999999999999e-05 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474610+0000", 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108606 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.474815+0000", 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020453800000000001 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.475681+0000", 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086620700000000002 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477079+0000", 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001398388 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477340+0000", 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026078800000000002 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477385+0000", 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5014e-05 2024-09-16T12:59:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:23.477419+0000", 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4143000000000002e-05 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:5580 4.12 e46/40 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head v 46'11, mlcod=46'9)", 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.105752067000001, 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039131280000000001, 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708994+0000", 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708997+0000", 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4109999999999999e-06 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710732+0000", 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001735002 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710766+0000", 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-05 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710779+0000", 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3137e-05 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710905+0000", 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012594799999999999 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710972+0000", 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7062000000000003e-05 2024-09-16T12:59:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.712815+0000", 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018428419999999999 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.712907+0000", 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2516000000000007e-05 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:7104 4.1c 4:394c1983:::rbd_data.11b58f77e1c4.0000000000000024:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.768236903999998, 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031870290000000001, 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7104 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046509+0000", 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046511+0000", 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9819999999999998e-06 2024-09-16T12:59:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046518+0000", 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4489999999999996e-06 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046536+0000", 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8544999999999999e-05 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046544+0000", 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.729e-06 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046624+0000", 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9708000000000003e-05 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.046755+0000", 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130846 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:33.049696+0000", 2024-09-16T12:59:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029417699999999998 2024-09-16T12:59:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15008 4.1c 4:3e8625c6:::rbd_data.11b5c2c64266.0000000000000162:head [set-alloc-hint object_size 524288 write_size 524288,write 32768~452] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:49.894991+0000", 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.919755007, 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041292409999999996, 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15008 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.894991+0000", 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.894991+0000", 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.894994+0000", 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895004+0000", 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0373e-05 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895028+0000", 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4193999999999999e-05 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895039+0000", 2024-09-16T12:59:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0268999999999999e-05 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895119+0000", 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0209000000000004e-05 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895289+0000", 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000169779 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895402+0000", 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113443 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.897278+0000", 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001876305 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.899011+0000", 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017325299999999999 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.899070+0000", 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9426999999999999e-05 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.899120+0000", 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0251999999999999e-05 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15071 4.1c 4:3ad94978:::rbd_data.11b5c2c64266.00000000000000ad:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:49.944671+0000", 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8700755500000001, 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031721560000000002, 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15071 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944671+0000", 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944671+0000", 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944673+0000", 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.153e-06 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944682+0000", 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1230000000000006e-06 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944707+0000", 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4746999999999999e-05 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944716+0000", 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4380000000000001e-06 2024-09-16T12:59:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944787+0000", 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0776000000000001e-05 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.944954+0000", 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016750000000000001 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.945648+0000", 2024-09-16T12:59:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00069389400000000002 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.947711+0000", 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020623659999999999 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.947773+0000", 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2126999999999997e-05 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.947806+0000", 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3277e-05 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.947843+0000", 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6755000000000003e-05 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15158 4.1c 4:3ad94978:::rbd_data.11b5c2c64266.00000000000000ad:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T12:59:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.044030+0000", 2024-09-16T12:59:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.7707157950000001, 2024-09-16T12:59:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035079220000000001, 2024-09-16T12:59:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15158 2024-09-16T12:59:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044030+0000", 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044030+0000", 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044033+0000", 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7039999999999999e-06 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044049+0000", 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5460000000000001e-05 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044083+0000", 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4493999999999999e-05 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044097+0000", 2024-09-16T12:59:56.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.38e-05 2024-09-16T12:59:56.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044190+0000", 2024-09-16T12:59:56.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2850999999999996e-05 2024-09-16T12:59:56.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044280+0000", 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0117999999999993e-05 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044401+0000", 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121586 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.045862+0000", 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014609849999999999 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.047464+0000", 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016013150000000001 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.047514+0000", 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0575999999999998e-05 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.047538+0000", 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4032999999999998e-05 2024-09-16T12:59:56.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:15322 4.d e65/31 4:b2b80615:::rbd_data.11b5c2c64266.0000000000000120:head v 65'86, mlcod=64'84)", 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.212817+0000", 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.6019297369999999, 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0097146730000000001, 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.212817+0000", 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.212817+0000", 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.212818+0000", 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.362e-06 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218072+0000", 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052537699999999996 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218092+0000", 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0290000000000001e-05 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218101+0000", 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0599999999999997e-06 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218159+0000", 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7840000000000002e-05 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218179+0000", 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9737000000000001e-05 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.222484+0000", 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043052580000000002 2024-09-16T12:59:56.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.222531+0000", 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7355999999999999e-05 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16687 4.1c 4:3e22654f:::rbd_data.11b5c2c64266.00000000000000c1:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.198609+0000", 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.6161377379999999, 2024-09-16T12:59:56.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029114599999999998, 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16687 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.198609+0000", 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.198609+0000", 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.198611+0000", 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.136e-06 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.198622+0000", 2024-09-16T12:59:56.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1266e-05 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.198649+0000", 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7492e-05 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.198663+0000", 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3872e-05 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.198746+0000", 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2921000000000001e-05 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.198894+0000", 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014731600000000001 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.199476+0000", 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00058284500000000002 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.201192+0000", 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017154760000000001 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.201439+0000", 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024720299999999998 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.201488+0000", 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8896000000000001e-05 2024-09-16T12:59:56.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.201520+0000", 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2036999999999998e-05 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16699 4.1c 4:3e22654f:::rbd_data.11b5c2c64266.00000000000000c1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.221905+0000", 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.5928410079999997, 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030538470000000002, 2024-09-16T12:59:56.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16699 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.221905+0000", 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.221905+0000", 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.221909+0000", 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5820000000000002e-06 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.221922+0000", 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3576e-05 2024-09-16T12:59:56.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.221965+0000", 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2885e-05 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.221982+0000", 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6869999999999999e-05 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.222103+0000", 2024-09-16T12:59:56.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120355 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.222315+0000", 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021238499999999999 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.222574+0000", 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025863600000000002 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.224420+0000", 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018462730000000001 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.224788+0000", 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0003678 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.224886+0000", 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8346999999999999e-05 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.224959+0000", 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3138000000000003e-05 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:16750 4.1c 4:38475728:::rbd_data.11b5c2c64266.0000000000000144:head [set-alloc-hint object_size 524288 write_size 524288,write 425984~6039] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.285124+0000", 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.5296225210000003, 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029458129999999998, 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.902 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16750 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.285124+0000", 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.285124+0000", 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.285126+0000", 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1940000000000001e-06 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.285141+0000", 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5158e-05 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.285161+0000", 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0227000000000001e-05 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.285171+0000", 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5270000000000004e-06 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.285240+0000", 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9113e-05 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.285439+0000", 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019885700000000001 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.285644+0000", 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000205338 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.286755+0000", 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001110806 2024-09-16T12:59:56.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.287948+0000", 2024-09-16T12:59:56.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001192774 2024-09-16T12:59:56.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.288018+0000", 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0631999999999997e-05 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.288070+0000", 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1187000000000003e-05 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.908 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17494 4.1c 4:38ecd3a4:::rbd_data.11b5c2c64266.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.696703+0000", 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.118042848, 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034590519999999998, 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17494 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696703+0000", 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696703+0000", 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696705+0000", 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.978e-06 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696719+0000", 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3346999999999999e-05 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696745+0000", 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6108000000000001e-05 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696756+0000", 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.081e-05 2024-09-16T12:59:56.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696832+0000", 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6229000000000005e-05 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696918+0000", 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5836999999999998e-05 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.697024+0000", 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010665199999999999 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.698454+0000", 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014298169999999999 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.699986+0000", 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001531534 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.700091+0000", 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105157 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.700162+0000", 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1582999999999995e-05 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17945 4.1c 4:396e1e85:::rbd_data.11b51a3eb68b.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 709338~32506] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T12:59:56.914 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:54.938617+0000", 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.8761287719999999, 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006632303, 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17945 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.938617+0000", 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.938617+0000", 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.938620+0000", 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2369999999999999e-06 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.939112+0000", 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049183400000000002 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.939144+0000", 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2629000000000002e-05 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.939156+0000", 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2094e-05 2024-09-16T12:59:56.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.939281+0000", 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012501400000000001 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.939716+0000", 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043464399999999998 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.940777+0000", 2024-09-16T12:59:56.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001061523 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.944819+0000", 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040418310000000001 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.944972+0000", 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015235300000000001 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.945139+0000", 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016715 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.945250+0000", 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110994 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:18066 4.1c 4:3fb01882:::rbd_data.11b51a3eb68b.0000000000000038:head [set-alloc-hint object_size 1048576 write_size 1048576,write 166190~54369] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-16T12:59:56.920 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:56.207045+0000", 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.60770092600000003, 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039314800000000002, 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18066 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207045+0000", 2024-09-16T12:59:56.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207045+0000", 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207049+0000", 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207128+0000", 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9149999999999999e-05 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207167+0000", 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9036999999999999e-05 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207181+0000", 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3842999999999999e-05 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207287+0000", 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106807 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207569+0000", 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028195 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207835+0000", 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026603599999999999 2024-09-16T12:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.210372+0000", 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025370089999999998 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.210808+0000", 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043526699999999998 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.210904+0000", 2024-09-16T12:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6737999999999994e-05 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.210977+0000", 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2449999999999999e-05 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:18078 4.1c 4:3b3eadb4:::rbd_data.11b51a3eb68b.00000000000000d5:head [set-alloc-hint object_size 1048576 write_size 1048576,write 478493~45795] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:56.230483+0000", 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.58426365899999999, 2024-09-16T12:59:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031121030000000002, 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18078 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.230483+0000", 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.230483+0000", 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.230485+0000", 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7460000000000001e-06 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.230521+0000", 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5966000000000001e-05 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.230551+0000", 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.006e-05 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.230562+0000", 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0706e-05 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.230675+0000", 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011262199999999999 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.230924+0000", 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024890300000000003 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.231172+0000", 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024847600000000002 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.232942+0000", 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017699669999999999 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.233473+0000", 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00053109700000000001 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.233529+0000", 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5813999999999998e-05 2024-09-16T12:59:56.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.233595+0000", 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5746000000000004e-05 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:18092 4.1c 4:3fb01882:::rbd_data.11b51a3eb68b.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:56.266119+0000", 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.54862758499999997, 2024-09-16T12:59:56.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003767355, 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18092 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266119+0000", 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266119+0000", 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266122+0000", 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2629999999999999e-06 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266138+0000", 2024-09-16T12:59:56.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6203000000000001e-05 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266185+0000", 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7207999999999999e-05 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266204+0000", 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8709e-05 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266285+0000", 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0517000000000002e-05 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266370+0000", 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5574000000000004e-05 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266500+0000", 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012964500000000001 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.267799+0000", 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00129962 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.269803+0000", 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020033479999999998 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.269851+0000", 2024-09-16T12:59:56.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8557999999999998e-05 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.269886+0000", 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4709999999999998e-05 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:56.938 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:56.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T12:59:56.940 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:56.940 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:56.941 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:56.934+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:56.957 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:56.976 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:56.998 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:57.021 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:57.041 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:57.031+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:57.043 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:57.048 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:57.048 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:57.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T12:59:57.063 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:57.065 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:57.065 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:57.065 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:57.066 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:57.066 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:57.066 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:57.066 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:57.066 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:57.095 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T12:59:57.095 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T12:59:57.095 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-16T12:59:57.095 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T12:59:57.095 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T12:59:57.096 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-16T12:59:57.142 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:57.138+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:57.172 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:57.172 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:57.184 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T12:59:57.209 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T12:59:57.210 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "" 2024-09-16T12:59:57.210 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T12:59:57.231 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone17 from 0xdd537af (overlap 0xc7c65) to 0x9fd1d 2024-09-16T12:59:57.242 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:57.235+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:57.294 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:57.294 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:57.304 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T12:59:57.338 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-16T12:59:57.342 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:57.338+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:57.415 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:57.416 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:57.425 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T12:59:57.443 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:57.435+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:57.539 INFO:teuthology.orchestra.run.smithi042.stdout:2515 read 0x68e1e9c thru 0x68e5f30 (0x4095 bytes) 2024-09-16T12:59:57.540 INFO:teuthology.orchestra.run.smithi042.stdout:2516 punch from 0xa535244 to 0xa538728, (0x34e4 bytes) 2024-09-16T12:59:57.545 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:57.535+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:57.548 INFO:teuthology.orchestra.run.smithi042.stdout:2517 read 0xcd962f4 thru 0xcd991e4 (0x2ef1 bytes) 2024-09-16T12:59:57.548 INFO:teuthology.orchestra.run.smithi042.stdout:2518 write 0x4fa04aa thru 0x4fad17f (0xccd6 bytes) 2024-09-16T12:59:57.556 INFO:teuthology.orchestra.run.smithi042.stdout:2519 punch from 0x8aca56e to 0x8acc7e0, (0x2272 bytes) 2024-09-16T12:59:57.556 INFO:teuthology.orchestra.run.smithi042.stdout:2520 write 0x52099d0 thru 0x52118fd (0x7f2e bytes) 2024-09-16T12:59:57.564 INFO:teuthology.orchestra.run.smithi042.stdout:2521 write 0x773f242 thru 0x774ec2e (0xf9ed bytes) 2024-09-16T12:59:57.574 INFO:teuthology.orchestra.run.smithi042.stdout:2522 writesame 0xcfe2d6c thru 0xcfeb9be data_size 0x1(0x8c53 bytes) 2024-09-16T12:59:57.578 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:57.578 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:57.588 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T12:59:57.605 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.605 INFO:teuthology.orchestra.run.smithi042.stdout:2526 read 0xaabd3ed thru 0xaabdb4f (0x763 bytes) 2024-09-16T12:59:57.605 INFO:teuthology.orchestra.run.smithi042.stdout:2527 punch from 0x2d92199 to 0x2d98449, (0x62b0 bytes) 2024-09-16T12:59:57.605 INFO:teuthology.orchestra.run.smithi042.stdout:2528 read 0x9447d52 thru 0x9448403 (0x6b2 bytes) 2024-09-16T12:59:57.605 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.605 INFO:teuthology.orchestra.run.smithi042.stdout:2530 write 0x6d4d89b thru 0x6d55cf4 (0x845a bytes) 2024-09-16T12:59:57.612 INFO:teuthology.orchestra.run.smithi042.stdout:2531 read 0x6a1294f thru 0x6a16a59 (0x410b bytes) 2024-09-16T12:59:57.612 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.612 INFO:teuthology.orchestra.run.smithi042.stdout:2534 read 0x20d6906 thru 0x20d980b (0x2f06 bytes) 2024-09-16T12:59:57.612 INFO:teuthology.orchestra.run.smithi042.stdout:2535 read 0x4b71c00 thru 0x4b7b4f2 (0x98f3 bytes) 2024-09-16T12:59:57.613 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.638 INFO:teuthology.orchestra.run.smithi042.stdout:2538 trunc from 0xdd537af to 0x19d0ffc 2024-09-16T12:59:57.645 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:57.635+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:57.692 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:57.692 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:57.702 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T12:59:57.708 INFO:teuthology.orchestra.run.smithi042.stdout:2539 write 0xaaba20e thru 0xaac00a1 (0x5e94 bytes) 2024-09-16T12:59:57.718 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.718 INFO:teuthology.orchestra.run.smithi042.stdout:2541 punch from 0x7ffea9b to 0x80035ad, (0x4b12 bytes) 2024-09-16T12:59:57.718 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.718 INFO:teuthology.orchestra.run.smithi042.stdout:2544 writesame 0x5d48340 thru 0x5d4ed4c data_size 0x1(0x6a0d bytes) 2024-09-16T12:59:57.745 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.746 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:57.739+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:57.761 INFO:teuthology.orchestra.run.smithi042.stdout:2546 write 0xc844f4b thru 0xc848adb (0x3b91 bytes) 2024-09-16T12:59:57.768 INFO:teuthology.orchestra.run.smithi042.stdout:2548 punch from 0x832c37f to 0x832c7ca, (0x44b bytes) 2024-09-16T12:59:57.768 INFO:teuthology.orchestra.run.smithi042.stdout:2549 writesame 0x79320d3 thru 0x793a78a data_size 0x9(0x86b8 bytes) 2024-09-16T12:59:57.777 INFO:teuthology.orchestra.run.smithi042.stdout:2552 write 0xb98ed09 thru 0xb99c095 (0xd38d bytes) 2024-09-16T12:59:57.784 INFO:teuthology.orchestra.run.smithi042.stdout:2554 write 0x7cef2f7 thru 0x7cf38ef (0x45f9 bytes) 2024-09-16T12:59:57.790 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.791 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.801 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:57.801 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:57.811 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T12:59:57.814 INFO:teuthology.orchestra.run.smithi042.stdout:2559 trunc from 0xc848adc to 0x238ba9f 2024-09-16T12:59:57.839 INFO:teuthology.orchestra.run.smithi042.stdout:2561 read 0x174ecdf thru 0x1757812 (0x8b34 bytes) 2024-09-16T12:59:57.846 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:57.839+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:57.859 INFO:teuthology.orchestra.run.smithi042.stdout:2562 write 0x52ac7fe thru 0x52b264f (0x5e52 bytes) 2024-09-16T12:59:57.874 INFO:teuthology.orchestra.run.smithi042.stdout:2564 trunc from 0x52b2650 to 0x23d5b40 2024-09-16T12:59:57.909 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:57.909 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:57.920 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T12:59:57.925 INFO:teuthology.orchestra.run.smithi042.stdout:2566 writesame 0x943e7ab thru 0x944959c data_size 0x7a(0xadf2 bytes) 2024-09-16T12:59:57.925 INFO:teuthology.orchestra.run.smithi042.stdout:2567 read 0xf0bd7b thru 0xf16020 (0xa2a6 bytes) 2024-09-16T12:59:57.926 INFO:teuthology.orchestra.run.smithi042.stdout:2569 write 0x1dce004 thru 0x1dd8ab6 (0xaab3 bytes) 2024-09-16T12:59:57.942 INFO:teuthology.orchestra.run.smithi042.stdout:2570 writesame 0xa25bbb4 thru 0xa25f65c data_size 0x1(0x3aa9 bytes) 2024-09-16T12:59:57.942 INFO:teuthology.orchestra.run.smithi042.stdout:2571 read 0xe1e371 thru 0xe27721 (0x93b1 bytes) 2024-09-16T12:59:57.944 INFO:teuthology.orchestra.run.smithi042.stdout:2574 write 0x7eefc5d thru 0x7eff715 (0xfab9 bytes) 2024-09-16T12:59:57.947 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T12:59:57.942+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T12:59:57.969 INFO:teuthology.orchestra.run.smithi042.stdout:2576 write 0xca22e4f thru 0xca32e17 (0xffc9 bytes) 2024-09-16T12:59:57.979 INFO:teuthology.orchestra.run.smithi042.stdout:2577 trunc from 0xca32e18 to 0xdc25179 2024-09-16T12:59:57.992 INFO:teuthology.orchestra.run.smithi042.stdout:2579 read 0x3d0e8e6 thru 0x3d10d96 (0x24b1 bytes) 2024-09-16T12:59:57.992 INFO:teuthology.orchestra.run.smithi042.stdout:2581 read 0x78032c6 thru 0x7808e7f (0x5bba bytes) 2024-09-16T12:59:57.992 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.993 INFO:teuthology.orchestra.run.smithi042.stdout:2585 punch from 0x4e62bd5 to 0x4e6e7b8, (0xbbe3 bytes) 2024-09-16T12:59:57.993 INFO:teuthology.orchestra.run.smithi042.stdout:2586 writesame 0xa92a113 thru 0xa92bb59 data_size 0x7(0x1a47 bytes) 2024-09-16T12:59:57.999 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:57.999 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:58.000 INFO:teuthology.orchestra.run.smithi042.stdout:2590 writesame 0xb01196c thru 0xb012a99 data_size 0x3(0x112e bytes) 2024-09-16T12:59:58.007 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:58.007 INFO:teuthology.orchestra.run.smithi042.stdout:2593 write 0x6965eef thru 0x696fcd7 (0x9de9 bytes) 2024-09-16T12:59:58.013 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T12:59:58.016 INFO:teuthology.orchestra.run.smithi042.stdout:2598 trunc from 0xdc25179 to 0xead5f7b 2024-09-16T12:59:58.023 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:58.023 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:58.031 INFO:teuthology.orchestra.run.smithi042.stdout:2599 read 0x983e8ed thru 0x984c587 (0xdc9b bytes) 2024-09-16T12:59:58.031 INFO:teuthology.orchestra.run.smithi042.stdout:2602 read 0x982f11e thru 0x9837115 (0x7ff8 bytes) 2024-09-16T12:59:58.031 INFO:teuthology.orchestra.run.smithi042.stdout:2603 punch from 0x2c6260e to 0x2c6bc85, (0x9677 bytes) 2024-09-16T12:59:58.031 INFO:teuthology.orchestra.run.smithi042.stdout:2606 read 0xc563e86 thru 0xc57008a (0xc205 bytes) 2024-09-16T12:59:58.032 INFO:teuthology.orchestra.run.smithi042.stdout:2609 writesame 0x37f7937 thru 0x3806285 data_size 0x2b(0xe94f bytes) 2024-09-16T12:59:58.034 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T12:59:58.040 INFO:teuthology.orchestra.run.smithi042.stdout:2612 write 0x70564b8 thru 0x7060cec (0xa835 bytes) 2024-09-16T12:59:58.048 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:58.039+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:58.050 INFO:teuthology.orchestra.run.smithi042.stdout:2613 clone 19 order 18 su 4096 sc 7 2024-09-16T12:59:58.122 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T12:59:58.130 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:58.131 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:58.141 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T12:59:58.186 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T12:59:58.182+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T12:59:58.235 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:58.235 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:58.245 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T12:59:58.250 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T12:59:58.242+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T12:59:58.266 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone18 from 0xead5f7b (overlap 0x19d0ffc) to 0x0 2024-09-16T12:59:58.343 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:58.343 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:58.352 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T12:59:58.352 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:58.343+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:58.449 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:58.449 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:58.451 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:58.443+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:58.458 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T12:59:58.553 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:58.546+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:58.565 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:58.565 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:58.575 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T12:59:58.610 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-16T12:59:58.617 INFO:teuthology.orchestra.run.smithi042.stdout:2614 read 0xb893d6c thru 0xb89c6b7 (0x894c bytes) 2024-09-16T12:59:58.619 INFO:teuthology.orchestra.run.smithi042.stdout:2615 read 0x7fcb982 thru 0x7fd8010 (0xc68f bytes) 2024-09-16T12:59:58.620 INFO:teuthology.orchestra.run.smithi042.stdout:2617 read 0xc988b03 thru 0xc9928a6 (0x9da4 bytes) 2024-09-16T12:59:58.621 INFO:teuthology.orchestra.run.smithi042.stdout:2619 write 0x884e5a0 thru 0x885b8c3 (0xd324 bytes) 2024-09-16T12:59:58.653 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:58.643+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:58.654 INFO:teuthology.orchestra.run.smithi042.stdout:2620 punch from 0x82900a4 to 0x82942c6, (0x4222 bytes) 2024-09-16T12:59:58.654 INFO:teuthology.orchestra.run.smithi042.stdout:2621 write 0xc12eb11 thru 0xc12f979 (0xe69 bytes) 2024-09-16T12:59:58.665 INFO:teuthology.orchestra.run.smithi042.stdout:2622 write 0x5906858 thru 0x5909bc3 (0x336c bytes) 2024-09-16T12:59:58.677 INFO:teuthology.orchestra.run.smithi042.stdout:2624 clone 20 order 25 su 4194304 sc 14 2024-09-16T12:59:58.687 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:58.687 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:58.696 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T12:59:58.755 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:58.747+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:58.805 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:58.805 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:58.815 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T12:59:58.856 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:58.847+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:58.924 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T12:59:58.956 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:58.954+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:59.024 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:59.024 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:59.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T12:59:59.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T12:59:59.078 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T12:59:59.071+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T12:59:59.132 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T12:59:59.133 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T12:59:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T12:59:59.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T12:59:59.158 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:59.154+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:59.221 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T12:59:59.244 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T12:59:59.245 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:59.245 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:59.256 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T12:59:59.259 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:59.251+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:59.264 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T12:59:59.280 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone19 2024-09-16T12:59:59.284 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T12:59:59.321 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T12:59:59.361 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T12:59:59.363 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T12:59:59.355+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T12:59:59.410 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T12:59:59.411 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:59.411 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:59.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:59.411 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17061 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309089+0000", 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0968152819999997, 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017211035999999999, 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17061 2024-09-16T12:59:59.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309089+0000", 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309089+0000", 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309089+0000", 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2399999999999999e-07 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309091+0000", 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8050000000000001e-06 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309097+0000", 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0440000000000001e-06 2024-09-16T12:59:59.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309101+0000", 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6380000000000002e-06 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326190+0000", 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017088728000000001 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326247+0000", 2024-09-16T12:59:59.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7367000000000001e-05 2024-09-16T12:59:59.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326300+0000", 2024-09-16T12:59:59.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-05 2024-09-16T12:59:59.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.416 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.417 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17062 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.417 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309105+0000", 2024-09-16T12:59:59.417 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0967993749999998, 2024-09-16T12:59:59.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017328626999999999, 2024-09-16T12:59:59.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17062 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309105+0000", 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309105+0000", 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309105+0000", 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9700000000000002e-07 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309107+0000", 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309113+0000", 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8850000000000004e-06 2024-09-16T12:59:59.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309117+0000", 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5939999999999998e-06 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326330+0000", 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017213209 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326381+0000", 2024-09-16T12:59:59.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1523999999999997e-05 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326433+0000", 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2064999999999997e-05 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17063 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309120+0000", 2024-09-16T12:59:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0967838509999996, 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017433209000000002, 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17063 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309120+0000", 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309120+0000", 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309121+0000", 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309122+0000", 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7090000000000001e-06 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309128+0000", 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999997e-06 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309131+0000", 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3459999999999998e-06 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326465+0000", 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017333818000000001 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326503+0000", 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7848000000000001e-05 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326554+0000", 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0779000000000003e-05 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17064 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309135+0000", 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0967692630000005, 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017544561, 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17064 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309135+0000", 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309135+0000", 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309135+0000", 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9499999999999998e-07 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309137+0000", 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0150000000000002e-06 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309143+0000", 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9619999999999998e-06 2024-09-16T12:59:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309147+0000", 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6239999999999999e-06 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326582+0000", 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017434999 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326625+0000", 2024-09-16T12:59:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3473999999999997e-05 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326679+0000", 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4092000000000003e-05 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.430 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17065 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 393216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.431 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309157+0000", 2024-09-16T12:59:59.431 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0967467630000005, 2024-09-16T12:59:59.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017644177, 2024-09-16T12:59:59.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17065 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309157+0000", 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309157+0000", 2024-09-16T12:59:59.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309158+0000", 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9000000000000002e-07 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309160+0000", 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9e-06 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309166+0000", 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5710000000000003e-06 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309170+0000", 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7929999999999999e-06 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326711+0000", 2024-09-16T12:59:59.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017540561 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326749+0000", 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8438e-05 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326802+0000", 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2524000000000001e-05 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17066 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309174+0000", 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0967298749999994, 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017752123000000002, 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17066 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.436 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309174+0000", 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309174+0000", 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309175+0000", 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309177+0000", 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.012e-06 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309183+0000", 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9630000000000003e-06 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309186+0000", 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4180000000000001e-06 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326829+0000", 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017643058999999999 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326874+0000", 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5287000000000002e-05 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326926+0000", 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2030000000000002e-05 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17067 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 557056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309190+0000", 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0967141770000008, 2024-09-16T12:59:59.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017933083999999998, 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17067 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309190+0000", 2024-09-16T12:59:59.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309190+0000", 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309190+0000", 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309192+0000", 2024-09-16T12:59:59.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7799999999999999e-06 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309198+0000", 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.677e-06 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309201+0000", 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.411e-06 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.326956+0000", 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017754492 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327030+0000", 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4128999999999994e-05 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327123+0000", 2024-09-16T12:59:59.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.323e-05 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17068 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 630784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309205+0000", 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0966995980000007, 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018142325000000001, 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17068 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309205+0000", 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309205+0000", 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309205+0000", 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309213+0000", 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2300000000000008e-06 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309221+0000", 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.52e-06 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309225+0000", 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327152+0000", 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01792728 2024-09-16T12:59:59.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327228+0000", 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6197000000000002e-05 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327347+0000", 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118517 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17069 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309229+0000", 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0966752300000007, 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018308606000000002, 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17069 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.450 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309229+0000", 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309229+0000", 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309230+0000", 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2099999999999996e-07 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309233+0000", 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3419999999999999e-06 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309239+0000", 2024-09-16T12:59:59.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309243+0000", 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0230000000000003e-06 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327380+0000", 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018136876999999999 2024-09-16T12:59:59.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327491+0000", 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011083599999999999 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327538+0000", 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6956999999999998e-05 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17070 4.2 4:458ab0d8:::rbd_data.11b528bff1d7.0000000000000003:13 [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309245+0000", 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0966587319999999, 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018517074000000001, 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17070 2024-09-16T12:59:59.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309245+0000", 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309245+0000", 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309246+0000", 2024-09-16T12:59:59.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8000000000000002e-07 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309247+0000", 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.451e-06 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309250+0000", 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0769999999999999e-06 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309252+0000", 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0820000000000001e-06 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327572+0000", 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018319561000000002 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327667+0000", 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327763+0000", 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5482999999999994e-05 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17071 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309254+0000", 2024-09-16T12:59:59.459 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0966498940000005, 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018675461000000001, 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17071 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309254+0000", 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309254+0000", 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309255+0000", 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3999999999999998e-07 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309256+0000", 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2100000000000001e-06 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309259+0000", 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0309999999999999e-06 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309261+0000", 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-16T12:59:59.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327789+0000", 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018528258999999998 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327840+0000", 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1085000000000003e-05 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327930+0000", 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9683000000000002e-05 2024-09-16T12:59:59.463 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17072 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309263+0000", 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0966415420000004, 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018917891999999999, 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17072 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309263+0000", 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309263+0000", 2024-09-16T12:59:59.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309263+0000", 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309264+0000", 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.065e-06 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309267+0000", 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8100000000000002e-06 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309269+0000", 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8819999999999999e-06 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327973+0000", 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018703930000000001 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328065+0000", 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2188000000000006e-05 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328181+0000", 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115648 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17073 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309271+0000", 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0966335859999994, 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019262013000000001, 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17073 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309271+0000", 2024-09-16T12:59:59.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309271+0000", 2024-09-16T12:59:59.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309271+0000", 2024-09-16T12:59:59.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5400000000000002e-07 2024-09-16T12:59:59.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309272+0000", 2024-09-16T12:59:59.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.540000000000001e-07 2024-09-16T12:59:59.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309275+0000", 2024-09-16T12:59:59.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-16T12:59:59.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309277+0000", 2024-09-16T12:59:59.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.95e-06 2024-09-16T12:59:59.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328225+0000", 2024-09-16T12:59:59.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018948901000000001 2024-09-16T12:59:59.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328318+0000", 2024-09-16T12:59:59.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2245999999999997e-05 2024-09-16T12:59:59.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328533+0000", 2024-09-16T12:59:59.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021494099999999999 2024-09-16T12:59:59.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.475 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.476 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17074 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.476 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309278+0000", 2024-09-16T12:59:59.476 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0966258559999993, 2024-09-16T12:59:59.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019430942, 2024-09-16T12:59:59.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.478 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17074 2024-09-16T12:59:59.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.478 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309278+0000", 2024-09-16T12:59:59.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309278+0000", 2024-09-16T12:59:59.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309279+0000", 2024-09-16T12:59:59.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4900000000000002e-07 2024-09-16T12:59:59.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309280+0000", 2024-09-16T12:59:59.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.426e-06 2024-09-16T12:59:59.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309283+0000", 2024-09-16T12:59:59.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8619999999999999e-06 2024-09-16T12:59:59.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309285+0000", 2024-09-16T12:59:59.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-16T12:59:59.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328540+0000", 2024-09-16T12:59:59.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019254789000000001 2024-09-16T12:59:59.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328641+0000", 2024-09-16T12:59:59.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010133599999999999 2024-09-16T12:59:59.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328709+0000", 2024-09-16T12:59:59.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8353000000000001e-05 2024-09-16T12:59:59.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.486 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.487 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17075 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.487 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309287+0000", 2024-09-16T12:59:59.487 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0966176050000005, 2024-09-16T12:59:59.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019911412, 2024-09-16T12:59:59.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.487 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17075 2024-09-16T12:59:59.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.488 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309287+0000", 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309287+0000", 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309287+0000", 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6600000000000003e-07 2024-09-16T12:59:59.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309288+0000", 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0449999999999999e-06 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309291+0000", 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8459999999999999e-06 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309293+0000", 2024-09-16T12:59:59.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9719999999999999e-06 2024-09-16T12:59:59.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328763+0000", 2024-09-16T12:59:59.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019470357000000001 2024-09-16T12:59:59.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328982+0000", 2024-09-16T12:59:59.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021875700000000001 2024-09-16T12:59:59.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329198+0000", 2024-09-16T12:59:59.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216169 2024-09-16T12:59:59.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.493 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.493 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17076 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.494 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309295+0000", 2024-09-16T12:59:59.494 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0966092500000002, 2024-09-16T12:59:59.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020043858000000001, 2024-09-16T12:59:59.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17076 2024-09-16T12:59:59.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.495 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309295+0000", 2024-09-16T12:59:59.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309295+0000", 2024-09-16T12:59:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309295+0000", 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5199999999999998e-07 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309296+0000", 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0380000000000001e-06 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309299+0000", 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.926e-06 2024-09-16T12:59:59.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309301+0000", 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8929999999999999e-06 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329222+0000", 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019920617000000002 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329276+0000", 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4743000000000001e-05 2024-09-16T12:59:59.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329339+0000", 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2389000000000002e-05 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17077 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309303+0000", 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0966011590000004, 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020302292999999999, 2024-09-16T12:59:59.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.500 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.500 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.500 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17077 2024-09-16T12:59:59.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.500 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309303+0000", 2024-09-16T12:59:59.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309303+0000", 2024-09-16T12:59:59.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309303+0000", 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5600000000000002e-07 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309304+0000", 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6899999999999996e-07 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309307+0000", 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0089999999999999e-06 2024-09-16T12:59:59.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309309+0000", 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0990000000000001e-06 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329377+0000", 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020068004 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329477+0000", 2024-09-16T12:59:59.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100134 2024-09-16T12:59:59.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329605+0000", 2024-09-16T12:59:59.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012782200000000001 2024-09-16T12:59:59.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.505 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.506 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17078 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.506 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309311+0000", 2024-09-16T12:59:59.508 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0965928900000002, 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020446052999999999, 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17078 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309311+0000", 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309311+0000", 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309312+0000", 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4200000000000002e-07 2024-09-16T12:59:59.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309313+0000", 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105e-06 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309316+0000", 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8679999999999999e-06 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309317+0000", 2024-09-16T12:59:59.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8920000000000001e-06 2024-09-16T12:59:59.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329639+0000", 2024-09-16T12:59:59.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020322030000000001 2024-09-16T12:59:59.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329706+0000", 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6291000000000006e-05 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329757+0000", 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1625000000000001e-05 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.514 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17079 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 663552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.514 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309319+0000", 2024-09-16T12:59:59.514 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0965848830000002, 2024-09-16T12:59:59.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020674145000000001, 2024-09-16T12:59:59.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17079 2024-09-16T12:59:59.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.515 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309319+0000", 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309319+0000", 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309320+0000", 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3699999999999999e-07 2024-09-16T12:59:59.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309320+0000", 2024-09-16T12:59:59.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5099999999999998e-07 2024-09-16T12:59:59.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309323+0000", 2024-09-16T12:59:59.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8270000000000002e-06 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309325+0000", 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8899999999999999e-06 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329805+0000", 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020479430999999999 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329901+0000", 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6507000000000003e-05 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329993+0000", 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2301999999999998e-05 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17080 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309329+0000", 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0965755730000009, 2024-09-16T12:59:59.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020787672, 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17080 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309329+0000", 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309329+0000", 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309329+0000", 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3999999999999998e-07 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309330+0000", 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2600000000000001e-07 2024-09-16T12:59:59.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309333+0000", 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0410000000000002e-06 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309335+0000", 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0169999999999999e-06 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.330016+0000", 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020681102999999999 2024-09-16T12:59:59.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.330070+0000", 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4246999999999997e-05 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.330116+0000", 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6097999999999998e-05 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.523 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:59.525 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T12:59:59.526 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T12:59:59.526 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T12:59:59.526 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T12:59:59.527 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T12:59:59.527 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T12:59:59.527 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T12:59:59.527 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T12:59:59.527 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T12:59:59.527 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T12:59:59.527 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T12:59:59.515+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T12:59:59.625 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:59.615+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:59.716 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:59.716 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:59.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.716 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:31181 4.1c 4.a2b0c55c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.711812+0000", 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0013381459999999999, 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001366183, 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 31181 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.717 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.711812+0000", 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.711812+0000", 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.711812+0000", 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8200000000000001e-07 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.711814+0000", 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8920000000000001e-06 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.711820+0000", 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8529999999999997e-06 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.711824+0000", 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6160000000000002e-06 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:31242 4.1c 4.a2b0c55c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.712587+0000", 2024-09-16T12:59:59.720 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00056344000000000001, 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000674392, 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 31242 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712587+0000", 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712587+0000", 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712588+0000", 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.08e-06 2024-09-16T12:59:59.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712593+0000", 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4090000000000002e-06 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712605+0000", 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1759999999999999e-05 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712612+0000", 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:31243 4.1c 4.a2b0c55c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.712619+0000", 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00053131000000000001, 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070166900000000003, 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 31243 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712619+0000", 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712619+0000", 2024-09-16T12:59:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712619+0000", 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0100000000000005e-07 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712622+0000", 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4150000000000002e-06 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712629+0000", 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4959999999999999e-06 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712634+0000", 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5179999999999998e-06 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:31244 4.1c 4.a2b0c55c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.712639+0000", 2024-09-16T12:59:59.727 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00051142199999999998, 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00074100800000000003, 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 31244 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712639+0000", 2024-09-16T12:59:59.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712639+0000", 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712639+0000", 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712641+0000", 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0849999999999999e-06 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712647+0000", 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1340000000000001e-06 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712651+0000", 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0810000000000004e-06 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.730 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:31245 4.1c 4.a2b0c55c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.712656+0000", 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.000494284, 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00079228499999999999, 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 31245 2024-09-16T12:59:59.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712656+0000", 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712656+0000", 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712656+0000", 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7599999999999998e-07 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712658+0000", 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9649999999999998e-06 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712664+0000", 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8139999999999997e-06 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.712668+0000", 2024-09-16T12:59:59.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7459999999999998e-06 2024-09-16T12:59:59.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.734 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.734 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T12:59:59.734 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 5 2024-09-16T12:59:59.734 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:59.735 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T12:59:59.736 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T12:59:59.727+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T12:59:59.836 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:59.830+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:59.886 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:59.886 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T12:59:59.886 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T12:59:59.887 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T12:59:59.887 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T12:59:59.895 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T12:59:59.937 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T12:59:59.930+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T12:59:59.990 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T12:59:59.990 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T12:59:59.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T12:59:59.990 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T12:59:59.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.990 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:681 4.1c 4:3ed6ce4a:::rbd_data.11b59d02cc38.0000000000000093:head [set-alloc-hint object_size 1048576 write_size 1048576,write 119408~11664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T12:59:59.990 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:59.990 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.375449715000002, 2024-09-16T12:59:59.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035485299999999998, 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 681 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608561+0000", 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608564+0000", 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.146e-06 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608592+0000", 2024-09-16T12:59:59.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8585000000000002e-05 2024-09-16T12:59:59.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T12:59:59.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608635+0000", 2024-09-16T12:59:59.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2629000000000001e-05 2024-09-16T12:59:59.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T12:59:59.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608649+0000", 2024-09-16T12:59:59.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4270000000000001e-05 2024-09-16T12:59:59.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T12:59:59.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.608797+0000", 2024-09-16T12:59:59.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014800499999999999 2024-09-16T12:59:59.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609162+0000", 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036455100000000001 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.609411+0000", 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024895400000000002 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T12:59:59.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611008+0000", 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015968860000000001 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.611948+0000", 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094050999999999998 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612013+0000", 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5185999999999996e-05 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.612109+0000", 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5808000000000002e-05 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:810 4.1c 4:3f2255b6:::rbd_data.11b59d02cc38.00000000000000dc:head [set-alloc-hint object_size 1048576 write_size 1048576,write 443751~14451] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.087366185, 2024-09-16T12:59:59.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036011279999999999, 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 810 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896644+0000", 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896648+0000", 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6109999999999998e-06 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896701+0000", 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3168999999999999e-05 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T12:59:59.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896743+0000", 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1445e-05 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896758+0000", 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5591e-05 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.896878+0000", 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120256 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897013+0000", 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000134327 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.897221+0000", 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020804800000000001 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.899109+0000", 2024-09-16T13:00:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018886899999999999 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900084+0000", 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097495899999999996 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900178+0000", 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3264999999999996e-05 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.900245+0000", 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7767000000000004e-05 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:860 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [set-alloc-hint object_size 1048576 write_size 1048576,write 997016~9314] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.998275+0000", 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.985735056999999, 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037930170000000001, 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 860 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998275+0000", 2024-09-16T13:00:00.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998280+0000", 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0969999999999999e-06 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998307+0000", 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7804999999999999e-05 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998349+0000", 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1838000000000002e-05 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998364+0000", 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5194000000000001e-05 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998511+0000", 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014693900000000001 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.998912+0000", 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000401066 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.999376+0000", 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000463938 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001244+0000", 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018672389999999999 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.001920+0000", 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067617100000000004 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002008+0000", 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8342000000000002e-05 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.002068+0000", 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0387999999999998e-05 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:995 4.1c 4:3e61cf43:::rbd_data.11b59d02cc38.0000000000000028:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:18.307157+0000", 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.676853964000003, 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040928859999999996, 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 995 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.009 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307157+0000", 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307159+0000", 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9739999999999998e-06 2024-09-16T13:00:00.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307174+0000", 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4134000000000001e-05 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307211+0000", 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7385000000000002e-05 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307230+0000", 2024-09-16T13:00:00.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8961e-05 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307461+0000", 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230721 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.307710+0000", 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024951200000000002 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.308016+0000", 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030602199999999998 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.309601+0000", 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015845200000000001 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311094+0000", 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00149321 2024-09-16T13:00:00.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311180+0000", 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6142000000000003e-05 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:18.311249+0000", 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9305000000000001e-05 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.015 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:5580 4.12 e46/40 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head v 46'11, mlcod=46'9)", 2024-09-16T13:00:00.015 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.708994+0000", 2024-09-16T13:00:00.015 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.275016282999999, 2024-09-16T13:00:00.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039131280000000001, 2024-09-16T13:00:00.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.015 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708994+0000", 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708994+0000", 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.708997+0000", 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4109999999999999e-06 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710732+0000", 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001735002 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710766+0000", 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-05 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710779+0000", 2024-09-16T13:00:00.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3137e-05 2024-09-16T13:00:00.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710905+0000", 2024-09-16T13:00:00.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012594799999999999 2024-09-16T13:00:00.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.710972+0000", 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7062000000000003e-05 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.712815+0000", 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018428419999999999 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.712907+0000", 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2516000000000007e-05 2024-09-16T13:00:00.019 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15008 4.1c 4:3e8625c6:::rbd_data.11b5c2c64266.0000000000000162:head [set-alloc-hint object_size 524288 write_size 524288,write 32768~452] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:49.894991+0000", 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.089019222999999, 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041292409999999996, 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.020 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15008 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.894991+0000", 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.894991+0000", 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.894994+0000", 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895004+0000", 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0373e-05 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895028+0000", 2024-09-16T13:00:00.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4193999999999999e-05 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895039+0000", 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0268999999999999e-05 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895119+0000", 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0209000000000004e-05 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895289+0000", 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000169779 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.895402+0000", 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113443 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.897278+0000", 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001876305 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.899011+0000", 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017325299999999999 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.899070+0000", 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9426999999999999e-05 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:49.899120+0000", 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0251999999999999e-05 2024-09-16T13:00:00.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15158 4.1c 4:3ad94978:::rbd_data.11b5c2c64266.00000000000000ad:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.044030+0000", 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9399800109999994, 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035079220000000001, 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15158 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044030+0000", 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044030+0000", 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044033+0000", 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7039999999999999e-06 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044049+0000", 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5460000000000001e-05 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044083+0000", 2024-09-16T13:00:00.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4493999999999999e-05 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044097+0000", 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.38e-05 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044190+0000", 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2850999999999996e-05 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044280+0000", 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0117999999999993e-05 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.044401+0000", 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121586 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.045862+0000", 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014609849999999999 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.047464+0000", 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016013150000000001 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.047514+0000", 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0575999999999998e-05 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.047538+0000", 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4032999999999998e-05 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.031 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:15322 4.d e65/31 4:b2b80615:::rbd_data.11b5c2c64266.0000000000000120:head v 65'86, mlcod=64'84)", 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.212817+0000", 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7711939530000009, 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0097146730000000001, 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.212817+0000", 2024-09-16T13:00:00.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.212817+0000", 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.212818+0000", 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.362e-06 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218072+0000", 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052537699999999996 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218092+0000", 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0290000000000001e-05 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218101+0000", 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0599999999999997e-06 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218159+0000", 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7840000000000002e-05 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.218179+0000", 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9737000000000001e-05 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.222484+0000", 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043052580000000002 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.222531+0000", 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7355999999999999e-05 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17494 4.1c 4:38ecd3a4:::rbd_data.11b5c2c64266.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.696703+0000", 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2873070640000002, 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034590519999999998, 2024-09-16T13:00:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17494 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696703+0000", 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696703+0000", 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696705+0000", 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.978e-06 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696719+0000", 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3346999999999999e-05 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696745+0000", 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6108000000000001e-05 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696756+0000", 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.081e-05 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696832+0000", 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6229000000000005e-05 2024-09-16T13:00:00.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.696918+0000", 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5836999999999998e-05 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.697024+0000", 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010665199999999999 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.698454+0000", 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014298169999999999 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.699986+0000", 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001531534 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.700091+0000", 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105157 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.700162+0000", 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1582999999999995e-05 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17945 4.1c 4:396e1e85:::rbd_data.11b51a3eb68b.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 709338~32506] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:54.938617+0000", 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0453929879999997, 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.006632303, 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.042 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17945 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.938617+0000", 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.938617+0000", 2024-09-16T13:00:00.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.938620+0000", 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2369999999999999e-06 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.939112+0000", 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00049183400000000002 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.939144+0000", 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2629000000000002e-05 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.939156+0000", 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2094e-05 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.939281+0000", 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012501400000000001 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.939716+0000", 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043464399999999998 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.940777+0000", 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001061523 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.944819+0000", 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0040418310000000001 2024-09-16T13:00:00.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.944972+0000", 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015235300000000001 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.945139+0000", 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016715 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:54.945250+0000", 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110994 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:18066 4.1c 4:3fb01882:::rbd_data.11b51a3eb68b.0000000000000038:head [set-alloc-hint object_size 1048576 write_size 1048576,write 166190~54369] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:56.207045+0000", 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7769651419999999, 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039314800000000002, 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18066 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207045+0000", 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207045+0000", 2024-09-16T13:00:00.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207049+0000", 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207128+0000", 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9149999999999999e-05 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207167+0000", 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9036999999999999e-05 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207181+0000", 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3842999999999999e-05 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207287+0000", 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106807 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207569+0000", 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028195 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.207835+0000", 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026603599999999999 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.210372+0000", 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025370089999999998 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.210808+0000", 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043526699999999998 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.210904+0000", 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6737999999999994e-05 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.210977+0000", 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2449999999999999e-05 2024-09-16T13:00:00.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:18092 4.1c 4:3fb01882:::rbd_data.11b51a3eb68b.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:56.266119+0000", 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7178918009999999, 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003767355, 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18092 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266119+0000", 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266119+0000", 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266122+0000", 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2629999999999999e-06 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266138+0000", 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6203000000000001e-05 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266185+0000", 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7207999999999999e-05 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266204+0000", 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8709e-05 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266285+0000", 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0517000000000002e-05 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266370+0000", 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5574000000000004e-05 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.266500+0000", 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012964500000000001 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.267799+0000", 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00129962 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.269803+0000", 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020033479999999998 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.269851+0000", 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8557999999999998e-05 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.269886+0000", 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4709999999999998e-05 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.059 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:19792 4.1c 4:3aa30d45:::rbd_data.11b59afb69bb.0000000000000009:16 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.283707+0000", 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.70030371300000005, 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035130890000000001, 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 19792 2024-09-16T13:00:00.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283707+0000", 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283707+0000", 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283707+0000", 2024-09-16T13:00:00.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283709+0000", 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283715+0000", 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7470000000000002e-06 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283719+0000", 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8630000000000001e-06 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287027+0000", 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033081909999999998 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287122+0000", 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4658000000000001e-05 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287220+0000", 2024-09-16T13:00:00.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8146999999999994e-05 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:19793 4.1c 4:3aa30d45:::rbd_data.11b59afb69bb.0000000000000009:16 [read 40960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.283723+0000", 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.70028748100000004, 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037564030000000002, 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 19793 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283723+0000", 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283723+0000", 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283723+0000", 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3500000000000002e-07 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283726+0000", 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0870000000000001e-06 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283731+0000", 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7049999999999996e-06 2024-09-16T13:00:00.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283735+0000", 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287248+0000", 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035137329999999998 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287353+0000", 2024-09-16T13:00:00.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104314 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287479+0000", 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126644 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:19794 4.1c 4:3aa30d45:::rbd_data.11b59afb69bb.0000000000000009:16 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.283739+0000", 2024-09-16T13:00:00.068 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.70027145899999999, 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003973191, 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 19794 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283739+0000", 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283739+0000", 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283739+0000", 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.27e-07 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283741+0000", 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7570000000000001e-06 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283747+0000", 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8660000000000002e-06 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283751+0000", 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8149999999999999e-06 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287509+0000", 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037578630000000002 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287613+0000", 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104293 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287712+0000", 2024-09-16T13:00:00.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9270000000000003e-05 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:19795 4.1c 4:3aa30d45:::rbd_data.11b59afb69bb.0000000000000009:16 [read 98304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.283755+0000", 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.70025559299999995, 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042128169999999998, 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 19795 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283755+0000", 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283755+0000", 2024-09-16T13:00:00.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283755+0000", 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283757+0000", 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7889999999999999e-06 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283763+0000", 2024-09-16T13:00:00.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5589999999999996e-06 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283766+0000", 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2569999999999999e-06 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287741+0000", 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039748709999999996 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287856+0000", 2024-09-16T13:00:00.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115763 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.287968+0000", 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011121899999999999 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:19796 4.1c 4:3aa30d45:::rbd_data.11b59afb69bb.0000000000000009:16 [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.283772+0000", 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.70023879200000005, 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047388639999999998, 2024-09-16T13:00:00.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 19796 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283772+0000", 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283772+0000", 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283772+0000", 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283782+0000", 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9850000000000001e-06 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283788+0000", 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1680000000000001e-06 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.283792+0000", 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.288317+0000", 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0045248739999999999 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.288414+0000", 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7233999999999998e-05 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.288511+0000", 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6479999999999998e-05 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.081 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4533.0:20347 4.4 e73/69 4:20cf1352:::rbd_data.11b5b635ee69.0000000000000003:head v 73'141, mlcod=72'139)", 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.318429+0000", 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.665581482, 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041041300000000001, 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.318429+0000", 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.318429+0000", 2024-09-16T13:00:00.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.318429+0000", 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.320276+0000", 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001847123 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.320289+0000", 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2182e-05 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.320294+0000", 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7729999999999996e-06 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.320573+0000", 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027867000000000001 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.320595+0000", 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1824e-05 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.322506+0000", 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019112269999999999 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.322533+0000", 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7035000000000001e-05 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.085 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:23289 4.1c 4:3aa30d45:::rbd_data.11b59afb69bb.0000000000000009:16 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.427147+0000", 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.556863055, 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034567439999999999, 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 23289 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427147+0000", 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427147+0000", 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427148+0000", 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-16T13:00:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427150+0000", 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427157+0000", 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000003e-06 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427161+0000", 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9609999999999999e-06 2024-09-16T13:00:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.430477+0000", 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003315625 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.430539+0000", 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1933999999999994e-05 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.430604+0000", 2024-09-16T13:00:00.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5586000000000005e-05 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:23290 4.1c 4:3aa30d45:::rbd_data.11b59afb69bb.0000000000000009:16 [read 1167360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.427403+0000", 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.55660721899999999, 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034904609999999998, 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 23290 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427403+0000", 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427403+0000", 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427404+0000", 2024-09-16T13:00:00.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9400000000000004e-07 2024-09-16T13:00:00.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427408+0000", 2024-09-16T13:00:00.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4379999999999997e-06 2024-09-16T13:00:00.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427418+0000", 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3829999999999998e-06 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.427423+0000", 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3480000000000003e-06 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.430741+0000", 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033179350000000002 2024-09-16T13:00:00.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.430814+0000", 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3215999999999998e-05 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.430894+0000", 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9346999999999998e-05 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.095 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:00.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:00.097 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:00.090+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:00.198 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:00.194+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:00.225 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:00.225 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:00.225 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:00.225 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:00.236 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:00.299 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:00.294+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:00.346 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:00.346 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:00.346 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:00.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:00.399 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:00.394+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:00.470 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:00.470 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:00.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:00.470 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:00.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.470 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8579 4.11 4:895e0e12:::rbd_data.11b55eb65e78.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.742439+0000", 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.723317752, 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025774687000000001, 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8579 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742439+0000", 2024-09-16T13:00:00.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742439+0000", 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742443+0000", 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.473e-06 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742461+0000", 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8329999999999999e-05 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742504+0000", 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3098e-05 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742520+0000", 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5982999999999999e-05 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742635+0000", 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000114537 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.742782+0000", 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014705100000000001 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.749390+0000", 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066079370000000004 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.767807+0000", 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018417086999999999 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.767925+0000", 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011807300000000001 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.768098+0000", 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172876 2024-09-16T13:00:00.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.768214+0000", 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116242 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:8611 4.11 4:895e0e12:::rbd_data.11b55eb65e78.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:36.815402+0000", 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.650355223999998, 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927149, 2024-09-16T13:00:00.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8611 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815402+0000", 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815402+0000", 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815405+0000", 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8030000000000001e-06 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815416+0000", 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1741e-05 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815463+0000", 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6303e-05 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815474+0000", 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1189000000000001e-05 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815580+0000", 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010592899999999999 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.815765+0000", 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185095 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.821526+0000", 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057609820000000004 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845015+0000", 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023488841999999999 2024-09-16T13:00:00.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845048+0000", 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2913999999999997e-05 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845204+0000", 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015631600000000001 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:36.845329+0000", 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012503500000000001 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:13818 4.11 4:8dc8e4d5:::rbd_data.11b51cc7997f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:45.688547+0000", 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.777209791000001, 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.032774062999999999, 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.482 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 13818 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.688547+0000", 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.688547+0000", 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.688550+0000", 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5799999999999999e-06 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.689948+0000", 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013981040000000001 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.689981+0000", 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3479999999999998e-05 2024-09-16T13:00:00.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.689994+0000", 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2286000000000001e-05 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.690084+0000", 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9818e-05 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.690297+0000", 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021293899999999999 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.701762+0000", 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011465145 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721206+0000", 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019444110000000001 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721217+0000", 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1452e-05 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721263+0000", 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6097999999999998e-05 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:45.721321+0000", 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8050999999999997e-05 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15366 4.14 4:2baea869:::rbd_data.11b5c2c64266.0000000000000096:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.267941+0000", 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.197816556999999, 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015615755, 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15366 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.488 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267941+0000", 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267941+0000", 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267942+0000", 2024-09-16T13:00:00.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7150000000000001e-06 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267950+0000", 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3320000000000003e-06 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267974+0000", 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4618e-05 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.267984+0000", 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3840000000000003e-06 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.268045+0000", 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1011999999999998e-05 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.268133+0000", 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8713000000000003e-05 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.268230+0000", 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6569000000000003e-05 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.269729+0000", 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001499268 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.283483+0000", 2024-09-16T13:00:00.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013754096 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.283525+0000", 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1835999999999997e-05 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.283556+0000", 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1211999999999999e-05 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15516 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302699+0000", 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.163058056000001, 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015400624, 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15516 2024-09-16T13:00:00.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302699+0000", 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302699+0000", 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302699+0000", 2024-09-16T13:00:00.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-16T13:00:00.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302702+0000", 2024-09-16T13:00:00.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3520000000000001e-06 2024-09-16T13:00:00.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302708+0000", 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7130000000000002e-06 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302710+0000", 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7410000000000001e-06 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317919+0000", 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01520839 2024-09-16T13:00:00.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.317998+0000", 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8959000000000001e-05 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318100+0000", 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010204699999999999 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15517 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 401408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302713+0000", 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.163044032, 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015596560000000001, 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15517 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302713+0000", 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302713+0000", 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302713+0000", 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302715+0000", 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6279999999999999e-06 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302719+0000", 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8959999999999996e-06 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302721+0000", 2024-09-16T13:00:00.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5119999999999998e-06 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318131+0000", 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015409849999999999 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318214+0000", 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2273000000000003e-05 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318310+0000", 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6068000000000002e-05 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15518 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302724+0000", 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.163033183, 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015782292, 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15518 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302724+0000", 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302724+0000", 2024-09-16T13:00:00.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302724+0000", 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.41e-07 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302726+0000", 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.42e-06 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302730+0000", 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302732+0000", 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318345+0000", 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015612475000000001 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318423+0000", 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8694000000000002e-05 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318506+0000", 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2922000000000003e-05 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.507 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15519 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302738+0000", 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.163019342, 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015930841000000001, 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15519 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302738+0000", 2024-09-16T13:00:00.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302738+0000", 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302738+0000", 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302740+0000", 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302748+0000", 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8460000000000008e-06 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302752+0000", 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4710000000000001e-06 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318542+0000", 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015789257000000001 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318615+0000", 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3313999999999996e-05 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318669+0000", 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3693000000000002e-05 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.511 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15520 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302759+0000", 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.162997882000001, 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016018316000000001, 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15520 2024-09-16T13:00:00.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302759+0000", 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302759+0000", 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302760+0000", 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5300000000000004e-07 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302762+0000", 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6570000000000001e-06 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302771+0000", 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2770000000000005e-06 2024-09-16T13:00:00.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302774+0000", 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.92e-06 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318688+0000", 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015913882000000001 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318730+0000", 2024-09-16T13:00:00.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2710999999999998e-05 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318778+0000", 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7315999999999999e-05 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:15521 4.17 4:e9044cbd:::rbd_data.11b528bff1d7.0000000000000015:13 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:00.516 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:50.302805+0000", 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.162952383, 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016378622999999998, 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15521 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302805+0000", 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302805+0000", 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302806+0000", 2024-09-16T13:00:00.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0400000000000005e-07 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302809+0000", 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302816+0000", 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0740000000000004e-06 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.302821+0000", 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0470000000000001e-06 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318799+0000", 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015978470000000002 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.318851+0000", 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1343e-05 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:50.319183+0000", 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033269199999999999 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:18195 4.10 4:0f2992f2:::rbd_data.11b51a3eb68b.000000000000008d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:56.348914+0000", 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1168426739999999, 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037467268999999997, 2024-09-16T13:00:00.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18195 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.348914+0000", 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.348914+0000", 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.348916+0000", 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.348923+0000", 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7789999999999998e-06 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.348941+0000", 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7487e-05 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.348947+0000", 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2829999999999998e-06 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.349021+0000", 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4510000000000003e-05 2024-09-16T13:00:00.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.349211+0000", 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018920499999999999 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.362424+0000", 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013213603000000001 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.386222+0000", 2024-09-16T13:00:00.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023797624999999999 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.386240+0000", 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8059000000000001e-05 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.386318+0000", 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8195000000000006e-05 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.386382+0000", 2024-09-16T13:00:00.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3542999999999998e-05 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:28402 4.0 4:003caed5:::rbd_data.11b59afb69bb.0000000000000004:16 [read 2498560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.618171+0000", 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.84758624400000004, 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015619301, 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28402 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618171+0000", 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618171+0000", 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618171+0000", 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618173+0000", 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7439999999999999e-06 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618179+0000", 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7559999999999996e-06 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618182+0000", 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5109999999999999e-06 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.624154+0000", 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0059712189999999998 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.624231+0000", 2024-09-16T13:00:00.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7752999999999999e-05 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.633790+0000", 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0095589079999999996 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:28403 4.0 4:003caed5:::rbd_data.11b59afb69bb.0000000000000004:16 [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.618186+0000", 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.84757086100000001, 2024-09-16T13:00:00.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015850685999999999, 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28403 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618186+0000", 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618186+0000", 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618187+0000", 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1200000000000003e-07 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618189+0000", 2024-09-16T13:00:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2400000000000002e-06 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618197+0000", 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618201+0000", 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0640000000000004e-06 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.633831+0000", 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015630755999999999 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.633938+0000", 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106167 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634037+0000", 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9395000000000006e-05 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:28404 4.0 4:003caed5:::rbd_data.11b59afb69bb.0000000000000004:16 [read 2555904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.618205+0000", 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.84755195100000003, 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01603189, 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28404 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618205+0000", 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618205+0000", 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618206+0000", 2024-09-16T13:00:00.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2600000000000002e-07 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618208+0000", 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5859999999999999e-06 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618215+0000", 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5699999999999998e-06 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618218+0000", 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6090000000000001e-06 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634070+0000", 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015851358999999999 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634155+0000", 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5480000000000002e-05 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634237+0000", 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1760000000000003e-05 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:28405 4.0 4:003caed5:::rbd_data.11b59afb69bb.0000000000000004:16 [read 2584576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.618223+0000", 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.84753443500000003, 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016227827, 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28405 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618223+0000", 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618223+0000", 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618223+0000", 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.53e-07 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618225+0000", 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2390000000000001e-06 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618231+0000", 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3979999999999998e-06 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618234+0000", 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3950000000000001e-06 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634272+0000", 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016037666999999999 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634363+0000", 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1316000000000005e-05 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634451+0000", 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7459000000000004e-05 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:28406 4.0 4:003caed5:::rbd_data.11b59afb69bb.0000000000000004:16 [read 2613248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.618238+0000", 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.84751958000000005, 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016433716000000001, 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28406 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618238+0000", 2024-09-16T13:00:00.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618238+0000", 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618238+0000", 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.27e-07 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618240+0000", 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0679999999999999e-06 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618246+0000", 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9669999999999998e-06 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618250+0000", 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6169999999999998e-06 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634500+0000", 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016250442 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634586+0000", 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5810999999999995e-05 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634671+0000", 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5383999999999994e-05 2024-09-16T13:00:00.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:28416 4.10 4:0ab554d3:::rbd_data.11b59afb69bb.0000000000000007:16 [read 2506752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.618313+0000", 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.84744432400000003, 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016353056000000001, 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28416 2024-09-16T13:00:00.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618313+0000", 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618313+0000", 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618313+0000", 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.41e-07 2024-09-16T13:00:00.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618315+0000", 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.829e-06 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618323+0000", 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0949999999999996e-06 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618327+0000", 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0199999999999996e-06 2024-09-16T13:00:00.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634484+0000", 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016156897 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634570+0000", 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6036000000000001e-05 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634666+0000", 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5838e-05 2024-09-16T13:00:00.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:28417 4.10 4:0ab554d3:::rbd_data.11b59afb69bb.0000000000000007:16 [read 2535424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.618331+0000", 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.84742563999999998, 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016532160000000001, 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28417 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618331+0000", 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618331+0000", 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618332+0000", 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618334+0000", 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.855e-06 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618340+0000", 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618343+0000", 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6119999999999999e-06 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634699+0000", 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016355354999999999 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634777+0000", 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8359e-05 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634864+0000", 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6479999999999999e-05 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:28418 4.10 4:0ab554d3:::rbd_data.11b59afb69bb.0000000000000007:16 [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.618347+0000", 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.84740970800000004, 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016722727999999999, 2024-09-16T13:00:00.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28418 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618347+0000", 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618347+0000", 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618348+0000", 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618350+0000", 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9010000000000001e-06 2024-09-16T13:00:00.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618355+0000", 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5960000000000003e-06 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618359+0000", 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4470000000000002e-06 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634904+0000", 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016545048 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.634984+0000", 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0589999999999999e-05 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.635070+0000", 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5790000000000004e-05 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:28419 4.10 4:0ab554d3:::rbd_data.11b59afb69bb.0000000000000007:16 [read 2592768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.618385+0000", 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.84737246399999999, 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017412705000000001, 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28419 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.562 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618385+0000", 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618385+0000", 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618385+0000", 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:00:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618388+0000", 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5509999999999998e-06 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618395+0000", 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1350000000000003e-06 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.618399+0000", 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4100000000000001e-06 2024-09-16T13:00:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.635599+0000", 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017199486 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.635702+0000", 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103195 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.635797+0000", 2024-09-16T13:00:00.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5381e-05 2024-09-16T13:00:00.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:00.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:00.566 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:00.567 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:00:00.568 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:00.562+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:00.668 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:00.662+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:00.679 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:00.679 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:00.679 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:00.679 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:00.688 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:00:00.769 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:00.762+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:00.789 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:00.789 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:00.790 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:00.790 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:00.790 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:00.799 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:00:00.802 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-16T13:00:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:2625 write 0x8411125 thru 0x841cdb3 (0xbc8f bytes) 2024-09-16T13:00:00.848 INFO:teuthology.orchestra.run.smithi042.stdout:2626 trunc from 0xead5f7b to 0x4e7d80d 2024-09-16T13:00:00.870 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:00.863+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:00.898 INFO:teuthology.orchestra.run.smithi042.stdout:2627 read 0x39964e2 thru 0x399c506 (0x6025 bytes) 2024-09-16T13:00:00.899 INFO:teuthology.orchestra.run.smithi042.stdout:2629 writesame 0x4dce8f0 thru 0x4dda65f data_size 0x38(0xbd70 bytes) 2024-09-16T13:00:00.901 INFO:teuthology.orchestra.run.smithi042.stdout:2633 punch from 0x36e4f7d to 0x36f0a0d, (0xba90 bytes) 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:590 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.559431250000003, 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034258392999999998, 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.903 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 590 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:00.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339271+0000", 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339281+0000", 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6670000000000008e-06 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339307+0000", 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5907000000000001e-05 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339317+0000", 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0356999999999999e-05 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339412+0000", 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4248999999999998e-05 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339533+0000", 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012184900000000001 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.347489+0000", 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079559599999999998 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373423+0000", 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025933971 2024-09-16T13:00:00.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373435+0000", 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1924e-05 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373492+0000", 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6842999999999998e-05 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373528+0000", 2024-09-16T13:00:00.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5063 4.13 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22711298~51120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.589286452000003, 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111447293, 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.909 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5063 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309416+0000", 2024-09-16T13:00:00.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358861+0000", 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049444717999999999 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358893+0000", 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2279e-05 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358902+0000", 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4390000000000006e-06 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359062+0000", 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016014100000000001 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359330+0000", 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026734699999999997 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.362566+0000", 2024-09-16T13:00:00.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032358769999999999 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.417541+0000", 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054975345000000002 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420706+0000", 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031651380000000001 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420790+0000", 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4283999999999995e-05 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420862+0000", 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1138999999999998e-05 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.914 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5225 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.094131132000001, 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027003329999999999, 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5225 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804571+0000", 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6309999999999999e-06 2024-09-16T13:00:00.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804579+0000", 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804600+0000", 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1114000000000001e-05 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804613+0000", 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3040000000000001e-05 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804713+0000", 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9854999999999999e-05 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804844+0000", 2024-09-16T13:00:00.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130963 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.810955+0000", 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061105880000000001 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831495+0000", 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020540467 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831507+0000", 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831544+0000", 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7413000000000002e-05 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831573+0000", 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8836e-05 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.920 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5504 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.430427526000003, 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020881448, 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5504 2024-09-16T13:00:00.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468276+0000", 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8370000000000001e-06 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468290+0000", 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.383e-05 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468327+0000", 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468341+0000", 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.331e-05 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468439+0000", 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8746999999999995e-05 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468593+0000", 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015343999999999999 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.472806+0000", 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042129849999999998 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488823+0000", 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016017686999999999 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488936+0000", 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112074 2024-09-16T13:00:00.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489029+0000", 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3801000000000005e-05 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489155+0000", 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125248 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8354 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.617333456000001, 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016884742000000001, 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8354 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281370+0000", 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3159999999999999e-06 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281382+0000", 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281413+0000", 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1519000000000001e-05 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281435+0000", 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1812999999999999e-05 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292521+0000", 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011085915999999999 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292628+0000", 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107311 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293108+0000", 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047976600000000001 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297972+0000", 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048640200000000001 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298174+0000", 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020222 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:00.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298226+0000", 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2058000000000002e-05 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298252+0000", 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5743000000000001e-05 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8677 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:00.932 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.888742486000002, 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01642155, 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8677 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009959+0000", 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2899999999999999e-07 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009961+0000", 2024-09-16T13:00:00.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3760000000000002e-06 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009965+0000", 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009968+0000", 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1190000000000001e-06 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023570+0000", 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013601317999999999 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023695+0000", 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012563400000000001 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024191+0000", 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000495434 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026285+0000", 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020948360000000001 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026295+0000", 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8889999999999997e-06 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026338+0000", 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2253999999999999e-05 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026380+0000", 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2153000000000001e-05 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8678 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.888728233999998, 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020138508999999999, 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8678 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009973+0000", 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3000000000000002e-07 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009975+0000", 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4739999999999999e-06 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009980+0000", 2024-09-16T13:00:00.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4399999999999998e-06 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009983+0000", 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1209999999999998e-06 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025464+0000", 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015480918 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025536+0000", 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2212000000000006e-05 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025768+0000", 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000232186 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029964+0000", 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041957899999999996 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029981+0000", 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6634e-05 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030070+0000", 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9487999999999994e-05 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030111+0000", 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0915999999999999e-05 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.943 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8718 4.1b 4:dbb4ad27:::rbd_data.11b55eb65e78.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.812413273000001, 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028260378999999999, 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8718 2024-09-16T13:00:00.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086290+0000", 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1950000000000002e-06 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086300+0000", 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0458999999999999e-05 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086331+0000", 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1117e-05 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086344+0000", 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3191e-05 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086423+0000", 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8288999999999995e-05 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086530+0000", 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010720799999999999 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.091612+0000", 2024-09-16T13:00:00.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050817429999999997 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114274+0000", 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022662832000000001 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114386+0000", 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011155899999999999 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:00.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114475+0000", 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9043999999999997e-05 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114548+0000", 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2742000000000003e-05 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24293 4.1b 4:d9b96624:::rbd_data.11b59afb69bb.0000000000000000:16 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470599+0000", 2024-09-16T13:00:00.949 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.428102118, 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015978698, 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24293 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470599+0000", 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470599+0000", 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470599+0000", 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6e-07 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470603+0000", 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9349999999999996e-06 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470606+0000", 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1109999999999999e-06 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470609+0000", 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9179999999999998e-06 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486410+0000", 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015801116 2024-09-16T13:00:00.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486485+0000", 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5469000000000005e-05 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486577+0000", 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889000000000001e-05 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24294 4.1b 4:d9b96624:::rbd_data.11b59afb69bb.0000000000000000:16 [read 1413120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4280897770000001, 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016422685999999999, 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24294 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5499999999999999e-07 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470612+0000", 2024-09-16T13:00:00.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1790000000000001e-06 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470615+0000", 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0580000000000002e-06 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470618+0000", 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.103e-06 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486878+0000", 2024-09-16T13:00:00.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016260398999999998 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486954+0000", 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5763e-05 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487034+0000", 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9929e-05 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24327 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470764+0000", 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.427936565, 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015921840999999999, 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24327 2024-09-16T13:00:00.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470764+0000", 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470764+0000", 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470764+0000", 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470771+0000", 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2729999999999999e-06 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470777+0000", 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7599999999999999e-06 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470782+0000", 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.767e-06 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486598+0000", 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015815949999999999 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486641+0000", 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2586000000000002e-05 2024-09-16T13:00:00.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486686+0000", 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5141e-05 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24328 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1339392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4279139199999999, 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016079547999999999, 2024-09-16T13:00:00.962 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24328 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0699999999999998e-07 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470789+0000", 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9659999999999999e-06 2024-09-16T13:00:00.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470795+0000", 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4110000000000004e-06 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470798+0000", 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5499999999999999e-06 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486710+0000", 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015911536 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486790+0000", 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9912000000000003e-05 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486866+0000", 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6766000000000003e-05 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24329 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1368064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4278886820000001, 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016167826999999999, 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24329 2024-09-16T13:00:00.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9999999999999998e-07 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470815+0000", 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6900000000000001e-06 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470821+0000", 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9050000000000002e-06 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470823+0000", 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486891+0000", 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016068000999999998 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486933+0000", 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1711000000000001e-05 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486980+0000", 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6643e-05 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24330 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.427875072, 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016312199999999999, 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.971 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24330 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:00.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6199999999999999e-07 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470827+0000", 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.333e-06 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470831+0000", 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.348e-06 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470832+0000", 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.849e-06 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486993+0000", 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016160834999999998 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487060+0000", 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6748000000000005e-05 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487138+0000", 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7824999999999994e-05 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.975 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24331 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1425408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.427866375, 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016546297000000001, 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24331 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470835+0000", 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4999999999999999e-07 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470836+0000", 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2869999999999999e-06 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470839+0000", 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0479999999999999e-06 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470841+0000", 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0090000000000002e-06 2024-09-16T13:00:00.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487212+0000", 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016370609000000001 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487288+0000", 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6584999999999999e-05 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487381+0000", 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2509000000000005e-05 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24358 4.1b 4:d9b96624:::rbd_data.11b59afb69bb.0000000000000000:16 [read 1417216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4266856620000001, 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015994596, 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24358 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:00.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472016+0000", 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5199999999999996e-07 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472017+0000", 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4249999999999999e-06 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472021+0000", 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.596e-06 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472024+0000", 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5860000000000001e-06 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487784+0000", 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015760066999999999 2024-09-16T13:00:00.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487912+0000", 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012761900000000001 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488010+0000", 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7651000000000004e-05 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24392 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1343488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472302+0000", 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4263991549999999, 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015871322, 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24392 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472302+0000", 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472302+0000", 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472302+0000", 2024-09-16T13:00:00.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6499999999999999e-07 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472304+0000", 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9149999999999999e-06 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472308+0000", 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3789999999999999e-06 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472312+0000", 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0510000000000001e-06 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488000+0000", 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015688771000000001 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488077+0000", 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7015999999999996e-05 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488173+0000", 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5525000000000004e-05 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24393 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1372160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.426385928, 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016028144000000001, 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.989 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24393 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9900000000000001e-07 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472317+0000", 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.547e-06 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472321+0000", 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8970000000000001e-06 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472325+0000", 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7759999999999997e-06 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488206+0000", 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015881025 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488272+0000", 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5982000000000006e-05 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488343+0000", 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0518000000000005e-05 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24394 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4263725789999999, 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016119201, 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24394 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:00.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:00.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:00.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:00.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:00.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:00.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472329+0000", 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9500000000000003e-07 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472330+0000", 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6360000000000001e-06 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472334+0000", 2024-09-16T13:00:00.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7730000000000001e-06 2024-09-16T13:00:00.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:00.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472337+0000", 2024-09-16T13:00:00.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7420000000000002e-06 2024-09-16T13:00:00.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:00.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488362+0000", 2024-09-16T13:00:00.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016025094 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488405+0000", 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3438e-05 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488447+0000", 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2023000000000001e-05 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:00.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24395 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1429504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.426361438, 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016957005000000001, 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24395 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:00.999 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472340+0000", 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2099999999999997e-07 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472341+0000", 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4190000000000001e-06 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472345+0000", 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8139999999999998e-06 2024-09-16T13:00:01.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472348+0000", 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5780000000000001e-06 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489143+0000", 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016795244000000001 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489213+0000", 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0438000000000006e-05 2024-09-16T13:00:01.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489296+0000", 2024-09-16T13:00:01.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2991000000000006e-05 2024-09-16T13:00:01.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.003 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.003 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.003 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:01.005 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:01.006 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:00.998+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:01.007 INFO:teuthology.orchestra.run.smithi042.stdout:2634 writesame 0x695b334 thru 0x6965463 data_size 0x8(0xa130 bytes) 2024-09-16T13:00:01.007 INFO:teuthology.orchestra.run.smithi042.stdout:2635 trunc from 0x6965464 to 0xcbaccf8 2024-09-16T13:00:01.007 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.007 INFO:teuthology.orchestra.run.smithi042.stdout:2637 read 0x9767016 thru 0x9776386 (0xf371 bytes) 2024-09-16T13:00:01.007 INFO:teuthology.orchestra.run.smithi042.stdout:2639 read 0x1f5f3a0 thru 0x1f6a18a (0xadeb bytes) 2024-09-16T13:00:01.007 INFO:teuthology.orchestra.run.smithi042.stdout:2640 read 0x83ba85c thru 0x83c11c3 (0x6968 bytes) 2024-09-16T13:00:01.007 INFO:teuthology.orchestra.run.smithi042.stdout:2641 read 0x3515671 thru 0x35171c7 (0x1b57 bytes) 2024-09-16T13:00:01.007 INFO:teuthology.orchestra.run.smithi042.stdout:2642 write 0x88c26b thru 0x88e6d0 (0x2466 bytes) 2024-09-16T13:00:01.007 INFO:teuthology.orchestra.run.smithi042.stdout:2644 read 0xcbaba4b thru 0xcbaccf7 (0x12ad bytes) 2024-09-16T13:00:01.007 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:2646 read 0x772741e thru 0x772c51f (0x5102 bytes) 2024-09-16T13:00:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:2649 trunc from 0xcbaccf8 to 0xa669468 2024-09-16T13:00:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:2650 write 0x73f5f2d thru 0x7404c65 (0xed39 bytes) 2024-09-16T13:00:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:2651 read 0x3fdd54f thru 0x3fe24e3 (0x4f95 bytes) 2024-09-16T13:00:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:2652 write 0x1bec425 thru 0x1bf86b4 (0xc290 bytes) 2024-09-16T13:00:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:2655 write 0xa45b2e3 thru 0xa45eda2 (0x3ac0 bytes) 2024-09-16T13:00:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:2657 trunc from 0xa669468 to 0x55943b9 2024-09-16T13:00:01.027 INFO:teuthology.orchestra.run.smithi042.stdout:2660 write 0xeb716cc thru 0xeb71c9f (0x5d4 bytes) 2024-09-16T13:00:01.030 INFO:teuthology.orchestra.run.smithi042.stdout:2662 punch from 0xaacca71 to 0xaad61be, (0x974d bytes) 2024-09-16T13:00:01.030 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.037 INFO:teuthology.orchestra.run.smithi042.stdout:2664 trunc from 0xeb71ca0 to 0xaa88dee 2024-09-16T13:00:01.054 INFO:teuthology.orchestra.run.smithi042.stdout:2665 trunc from 0xaa88dee to 0xb450775 2024-09-16T13:00:01.062 INFO:teuthology.orchestra.run.smithi042.stdout:2666 read 0x17fb285 thru 0x18022f5 (0x7071 bytes) 2024-09-16T13:00:01.063 INFO:teuthology.orchestra.run.smithi042.stdout:2668 read 0x5e0feb6 thru 0x5e15cc5 (0x5e10 bytes) 2024-09-16T13:00:01.063 INFO:teuthology.orchestra.run.smithi042.stdout:2669 write 0x96a8a8d thru 0x96adb42 (0x50b6 bytes) 2024-09-16T13:00:01.067 INFO:teuthology.orchestra.run.smithi042.stdout:2670 writesame 0x108fc21 thru 0x1098bf2 data_size 0x2(0x8fd2 bytes) 2024-09-16T13:00:01.083 INFO:teuthology.orchestra.run.smithi042.stdout:2672 read 0x7835c32 thru 0x7839208 (0x35d7 bytes) 2024-09-16T13:00:01.084 INFO:teuthology.orchestra.run.smithi042.stdout:2675 write 0x4e3327c thru 0x4e335ac (0x331 bytes) 2024-09-16T13:00:01.087 INFO:teuthology.orchestra.run.smithi042.stdout:2676 read 0x450a469 thru 0x4515b26 (0xb6be bytes) 2024-09-16T13:00:01.087 INFO:teuthology.orchestra.run.smithi042.stdout:2677 punch from 0x2a95d13 to 0x2a9720d, (0x14fa bytes) 2024-09-16T13:00:01.087 INFO:teuthology.orchestra.run.smithi042.stdout:2678 read 0x7d23c7b thru 0x7d25eb6 (0x223c bytes) 2024-09-16T13:00:01.088 INFO:teuthology.orchestra.run.smithi042.stdout:2681 write 0x6c53d3e thru 0x6c5459d (0x860 bytes) 2024-09-16T13:00:01.090 INFO:teuthology.orchestra.run.smithi042.stdout:2682 writesame 0x6168415 thru 0x61699c0 data_size 0x26(0x15ac bytes) 2024-09-16T13:00:01.092 INFO:teuthology.orchestra.run.smithi042.stdout:2683 read 0x1558be3 thru 0x1560416 (0x7834 bytes) 2024-09-16T13:00:01.093 INFO:teuthology.orchestra.run.smithi042.stdout:2684 writesame 0x211ce39 thru 0x211e646 data_size 0x1(0x180e bytes) 2024-09-16T13:00:01.106 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:01.099+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:01.113 INFO:teuthology.orchestra.run.smithi042.stdout:2685 trunc from 0xb450775 to 0x5f935b6 2024-09-16T13:00:01.128 INFO:teuthology.orchestra.run.smithi042.stdout:2686 read 0x4c41553 thru 0x4c4511a (0x3bc8 bytes) 2024-09-16T13:00:01.129 INFO:teuthology.orchestra.run.smithi042.stdout:2687 punch from 0x46d9c05 to 0x46dc144, (0x253f bytes) 2024-09-16T13:00:01.139 INFO:teuthology.orchestra.run.smithi042.stdout:2689 write 0x69837b0 thru 0x699308c (0xf8dd bytes) 2024-09-16T13:00:01.153 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:01.153 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:01.153 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:01.153 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:01.154 INFO:teuthology.orchestra.run.smithi042.stdout:2690 write 0x836aaed thru 0x837494f (0x9e63 bytes) 2024-09-16T13:00:01.156 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T13:00:01.157 INFO:teuthology.orchestra.run.smithi042.stdout:2691 read 0x75d109b thru 0x75dfab5 (0xea1b bytes) 2024-09-16T13:00:01.158 INFO:teuthology.orchestra.run.smithi042.stdout:2692 read 0x53bee17 thru 0x53cafb3 (0xc19d bytes) 2024-09-16T13:00:01.159 INFO:teuthology.orchestra.run.smithi042.stdout:2693 writesame 0x4a00b1 thru 0x4ac339 data_size 0x1(0xc289 bytes) 2024-09-16T13:00:01.163 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:01.194 INFO:teuthology.orchestra.run.smithi042.stdout:2695 read 0x7324b28 thru 0x732ecfc (0xa1d5 bytes) 2024-09-16T13:00:01.207 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:01.202+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:01.210 INFO:teuthology.orchestra.run.smithi042.stdout:2696 write 0x9e1f407 thru 0x9e21c6a (0x2864 bytes) 2024-09-16T13:00:01.213 INFO:teuthology.orchestra.run.smithi042.stdout:2697 read 0xca14c thru 0xcaf45 (0xdfa bytes) 2024-09-16T13:00:01.214 INFO:teuthology.orchestra.run.smithi042.stdout:2699 write 0x1490b5c thru 0x14980ea (0x758f bytes) 2024-09-16T13:00:01.217 INFO:teuthology.orchestra.run.smithi042.stdout:2701 read 0x1328caa thru 0x133775c (0xeab3 bytes) 2024-09-16T13:00:01.218 INFO:teuthology.orchestra.run.smithi042.stdout:2702 read 0x496931d thru 0x496cacd (0x37b1 bytes) 2024-09-16T13:00:01.219 INFO:teuthology.orchestra.run.smithi042.stdout:2703 read 0x1b19087 thru 0x1b1ad91 (0x1d0b bytes) 2024-09-16T13:00:01.220 INFO:teuthology.orchestra.run.smithi042.stdout:2704 punch from 0x489d939 to 0x48a7805, (0x9ecc bytes) 2024-09-16T13:00:01.220 INFO:teuthology.orchestra.run.smithi042.stdout:2705 read 0x1e8b977 thru 0x1e964c0 (0xab4a bytes) 2024-09-16T13:00:01.221 INFO:teuthology.orchestra.run.smithi042.stdout:2708 read 0x1d3608c thru 0x1d39491 (0x3406 bytes) 2024-09-16T13:00:01.222 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.222 INFO:teuthology.orchestra.run.smithi042.stdout:2710 read 0x23a384d thru 0x23a885d (0x5011 bytes) 2024-09-16T13:00:01.223 INFO:teuthology.orchestra.run.smithi042.stdout:2711 punch from 0x2732368 to 0x2740201, (0xde99 bytes) 2024-09-16T13:00:01.223 INFO:teuthology.orchestra.run.smithi042.stdout:2712 punch from 0x9499fb6 to 0x949e539, (0x4583 bytes) 2024-09-16T13:00:01.223 INFO:teuthology.orchestra.run.smithi042.stdout:2713 read 0x745dc55 thru 0x7469012 (0xb3be bytes) 2024-09-16T13:00:01.223 INFO:teuthology.orchestra.run.smithi042.stdout:2714 writesame 0x6b9cb53 thru 0x6b9cd75 data_size 0x1(0x223 bytes) 2024-09-16T13:00:01.227 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.227 INFO:teuthology.orchestra.run.smithi042.stdout:2717 write 0x57a1b63 thru 0x57a94c8 (0x7966 bytes) 2024-09-16T13:00:01.243 INFO:teuthology.orchestra.run.smithi042.stdout:2718 write 0xbf2dafa thru 0xbf35cdd (0x81e4 bytes) 2024-09-16T13:00:01.246 INFO:teuthology.orchestra.run.smithi042.stdout:2719 writesame 0x962c6c1 thru 0x9638c98 data_size 0x34(0xc5d8 bytes) 2024-09-16T13:00:01.250 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.250 INFO:teuthology.orchestra.run.smithi042.stdout:2721 read 0x1d1d5b7 thru 0x1d1dc50 (0x69a bytes) 2024-09-16T13:00:01.260 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:01.261 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:01.261 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:01.261 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:01.261 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:01.264 INFO:teuthology.orchestra.run.smithi042.stdout:2722 trunc from 0xbf35cde to 0x4604e3e 2024-09-16T13:00:01.271 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:01.277 INFO:teuthology.orchestra.run.smithi042.stdout:2723 punch from 0x135b377 to 0x13619b4, (0x663d bytes) 2024-09-16T13:00:01.278 INFO:teuthology.orchestra.run.smithi042.stdout:2726 read 0x2165337 thru 0x216a8b4 (0x557e bytes) 2024-09-16T13:00:01.297 INFO:teuthology.orchestra.run.smithi042.stdout:2728 writesame 0xa7241e2 thru 0xa72beae data_size 0x1(0x7ccd bytes) 2024-09-16T13:00:01.299 INFO:teuthology.orchestra.run.smithi042.stdout:2729 read 0x78c3026 thru 0x78c9052 (0x602d bytes) 2024-09-16T13:00:01.300 INFO:teuthology.orchestra.run.smithi042.stdout:2730 write 0x10e026d thru 0x10ebc6f (0xba03 bytes) 2024-09-16T13:00:01.302 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.302 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.302 INFO:teuthology.orchestra.run.smithi042.stdout:2737 writesame 0x443322d thru 0x443b82b data_size 0x1(0x85ff bytes) 2024-09-16T13:00:01.308 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:01.302+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:01.338 INFO:teuthology.orchestra.run.smithi042.stdout:2738 write 0xa8b79d0 thru 0xa8bca3b (0x506c bytes) 2024-09-16T13:00:01.341 INFO:teuthology.orchestra.run.smithi042.stdout:2740 read 0x3fdf8e6 thru 0x3fe255c (0x2c77 bytes) 2024-09-16T13:00:01.343 INFO:teuthology.orchestra.run.smithi042.stdout:2741 punch from 0x7288a7 to 0x734070, (0xb7c9 bytes) 2024-09-16T13:00:01.343 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.343 INFO:teuthology.orchestra.run.smithi042.stdout:2744 read 0x5be06ff thru 0x5be8b95 (0x8497 bytes) 2024-09-16T13:00:01.343 INFO:teuthology.orchestra.run.smithi042.stdout:2745 punch from 0x1a8ad8b to 0x1a951f3, (0xa468 bytes) 2024-09-16T13:00:01.343 INFO:teuthology.orchestra.run.smithi042.stdout:2746 write 0x136c221 thru 0x1375a81 (0x9861 bytes) 2024-09-16T13:00:01.346 INFO:teuthology.orchestra.run.smithi042.stdout:2747 read 0x4f74ca8 thru 0x4f79ba5 (0x4efe bytes) 2024-09-16T13:00:01.347 INFO:teuthology.orchestra.run.smithi042.stdout:2748 writesame 0x82b1c2b thru 0x82bfaec data_size 0x1(0xdec2 bytes) 2024-09-16T13:00:01.366 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:01.366 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:01.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:01.367 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:01.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.367 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8419 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:01.367 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:01.367 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.958802054, 2024-09-16T13:00:01.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040293592000000003, 2024-09-16T13:00:01.367 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.367 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:01.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.368 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8419 2024-09-16T13:00:01.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.368 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:01.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:01.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402894+0000", 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402907+0000", 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402940+0000", 2024-09-16T13:00:01.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3436000000000003e-05 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402953+0000", 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2840000000000001e-05 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403104+0000", 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150712 2024-09-16T13:00:01.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403202+0000", 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8403e-05 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.411015+0000", 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078133079999999997 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443018+0000", 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032002348999999999 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443075+0000", 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7703e-05 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443141+0000", 2024-09-16T13:00:01.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5705999999999997e-05 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443185+0000", 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4152e-05 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:8419 4.7 e54/40 ondisk, result = 0)", 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.431024+0000", 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.930669501000001, 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012172095000000001, 2024-09-16T13:00:01.375 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431024+0000", 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431024+0000", 2024-09-16T13:00:01.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431025+0000", 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3820000000000001e-06 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431032+0000", 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4389999999999997e-06 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431054+0000", 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2008e-05 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431064+0000", 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-05 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443061+0000", 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011997017 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443069+0000", 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7309999999999993e-06 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443196+0000", 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012701800000000001 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8612 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.847921+0000", 2024-09-16T13:00:01.379 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.51377239, 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039491146999999997, 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8612 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847921+0000", 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847921+0000", 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847924+0000", 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0199999999999999e-06 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847936+0000", 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1579999999999999e-05 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847968+0000", 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2033000000000003e-05 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847986+0000", 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7839e-05 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.848125+0000", 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013962800000000001 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.848277+0000", 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015126199999999999 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.856740+0000", 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084638479999999995 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887229+0000", 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030488491 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887253+0000", 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4409e-05 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887347+0000", 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3467999999999993e-05 2024-09-16T13:00:01.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887412+0000", 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5568999999999995e-05 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:15878 4.15 4:af50264e:::rbd_data.11b5c2c64266.0000000000000129:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.552855+0000", 2024-09-16T13:00:01.385 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.808838224, 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011857385, 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 15878 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552855+0000", 2024-09-16T13:00:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552855+0000", 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552858+0000", 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2960000000000001e-06 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552869+0000", 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1197e-05 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552902+0000", 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3219000000000002e-05 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552915+0000", 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.257e-05 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.553032+0000", 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117784 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.553314+0000", 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028161000000000001 2024-09-16T13:00:01.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.555090+0000", 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001776047 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.563585+0000", 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084951169999999999 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:01.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.564623+0000", 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001037638 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.564683+0000", 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9889999999999997e-05 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.564713+0000", 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0017000000000001e-05 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.391 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16273 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754081+0000", 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607612891, 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010285459, 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.392 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16273 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754081+0000", 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754081+0000", 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754081+0000", 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3000000000000001e-07 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754084+0000", 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.525e-06 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754091+0000", 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7319999999999998e-06 2024-09-16T13:00:01.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754096+0000", 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7099999999999998e-06 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764261+0000", 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010164566999999999 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764304+0000", 2024-09-16T13:00:01.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3316000000000003e-05 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764366+0000", 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2179000000000002e-05 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16274 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 294912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.396 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754101+0000", 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607592695999999, 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010382317, 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16274 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754101+0000", 2024-09-16T13:00:01.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754101+0000", 2024-09-16T13:00:01.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754102+0000", 2024-09-16T13:00:01.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2099999999999996e-07 2024-09-16T13:00:01.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754105+0000", 2024-09-16T13:00:01.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9950000000000001e-06 2024-09-16T13:00:01.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754113+0000", 2024-09-16T13:00:01.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4579999999999993e-06 2024-09-16T13:00:01.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754118+0000", 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.843e-06 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764384+0000", 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010266566 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764428+0000", 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3377999999999997e-05 2024-09-16T13:00:01.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764483+0000", 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5455999999999999e-05 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16275 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754123+0000", 2024-09-16T13:00:01.402 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607570814000001, 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010473434, 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16275 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754123+0000", 2024-09-16T13:00:01.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754123+0000", 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754123+0000", 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7300000000000001e-07 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754139+0000", 2024-09-16T13:00:01.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5458999999999999e-05 2024-09-16T13:00:01.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754147+0000", 2024-09-16T13:00:01.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6109999999999994e-06 2024-09-16T13:00:01.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754153+0000", 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6400000000000002e-06 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764501+0000", 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010347656 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764543+0000", 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2088000000000001e-05 2024-09-16T13:00:01.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764596+0000", 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3507000000000001e-05 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16276 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754158+0000", 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607535197000001, 2024-09-16T13:00:01.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010575052999999999, 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16276 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754158+0000", 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754158+0000", 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754159+0000", 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.44e-07 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754163+0000", 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9360000000000001e-06 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754171+0000", 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7470000000000005e-06 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754177+0000", 2024-09-16T13:00:01.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4749999999999998e-06 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764614+0000", 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010436543 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764655+0000", 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1628999999999997e-05 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764733+0000", 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8178999999999998e-05 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16277 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754182+0000", 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607511318, 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010745922, 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.412 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16277 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754182+0000", 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754182+0000", 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754183+0000", 2024-09-16T13:00:01.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0900000000000002e-07 2024-09-16T13:00:01.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754186+0000", 2024-09-16T13:00:01.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3440000000000001e-06 2024-09-16T13:00:01.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754194+0000", 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1729999999999995e-06 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754199+0000", 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0010000000000001e-06 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764784+0000", 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010584748 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764846+0000", 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2292000000000006e-05 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764928+0000", 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1854999999999995e-05 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.416 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16295 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 794624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754204+0000", 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607489344999999, 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010835667, 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.417 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16295 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754204+0000", 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754204+0000", 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754205+0000", 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9999999999999996e-07 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754208+0000", 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.492e-06 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754216+0000", 2024-09-16T13:00:01.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9859999999999994e-06 2024-09-16T13:00:01.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754221+0000", 2024-09-16T13:00:01.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3530000000000002e-06 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764947+0000", 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010725656 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764994+0000", 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256000000000003e-05 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765040+0000", 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6023999999999998e-05 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16296 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754246+0000", 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607447872, 2024-09-16T13:00:01.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011016732, 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16296 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754246+0000", 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754246+0000", 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754246+0000", 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6300000000000005e-07 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754249+0000", 2024-09-16T13:00:01.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2900000000000001e-06 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754254+0000", 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8270000000000003e-06 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754258+0000", 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765106+0000", 2024-09-16T13:00:01.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010848343 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765184+0000", 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8220000000000007e-05 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765262+0000", 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8039000000000003e-05 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16297 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 942080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754263+0000", 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607431037, 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011169742, 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16297 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754263+0000", 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754263+0000", 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754263+0000", 2024-09-16T13:00:01.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754265+0000", 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.798e-06 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754269+0000", 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0659999999999997e-06 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754272+0000", 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1370000000000002e-06 2024-09-16T13:00:01.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765310+0000", 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011038031 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765372+0000", 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1989999999999994e-05 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765432+0000", 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0371999999999997e-05 2024-09-16T13:00:01.430 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16298 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754275+0000", 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607418931, 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011358536000000001, 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16298 2024-09-16T13:00:01.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754275+0000", 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754275+0000", 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754275+0000", 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9999999999999998e-07 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754277+0000", 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4780000000000001e-06 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754281+0000", 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0820000000000001e-06 2024-09-16T13:00:01.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754284+0000", 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9050000000000001e-06 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765506+0000", 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01122248 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765572+0000", 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5458999999999998e-05 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765633+0000", 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1631999999999995e-05 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16299 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.435 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754296+0000", 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.6073977, 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011521811, 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16299 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754296+0000", 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754296+0000", 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754296+0000", 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.96e-07 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754298+0000", 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754303+0000", 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1420000000000001e-06 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754307+0000", 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1470000000000001e-06 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765675+0000", 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011368417 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765737+0000", 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1586000000000003e-05 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765818+0000", 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1193999999999995e-05 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16300 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754309+0000", 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607384153, 2024-09-16T13:00:01.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011692226999999999, 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16300 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754309+0000", 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754309+0000", 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754310+0000", 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2800000000000003e-07 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754311+0000", 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T13:00:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754316+0000", 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1540000000000004e-06 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754318+0000", 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7939999999999998e-06 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765846+0000", 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011527720999999999 2024-09-16T13:00:01.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765909+0000", 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3056000000000001e-05 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766002+0000", 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2588000000000002e-05 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16301 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754321+0000", 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607372736, 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011860290000000001, 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16301 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754321+0000", 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754321+0000", 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754321+0000", 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2099999999999998e-07 2024-09-16T13:00:01.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754323+0000", 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4139999999999999e-06 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754327+0000", 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9820000000000002e-06 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754329+0000", 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.582e-06 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766038+0000", 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011708435999999999 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766099+0000", 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1699999999999995e-05 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766181+0000", 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1854999999999995e-05 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16302 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754332+0000", 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.60736125, 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012047413999999999, 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.449 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16302 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754332+0000", 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754332+0000", 2024-09-16T13:00:01.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754333+0000", 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.39e-07 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754334+0000", 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3570000000000001e-06 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754338+0000", 2024-09-16T13:00:01.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.233e-06 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754341+0000", 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7240000000000001e-06 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766224+0000", 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011882547 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766291+0000", 2024-09-16T13:00:01.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7347000000000004e-05 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766380+0000", 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8867000000000001e-05 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16303 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754344+0000", 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607349964000001, 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012220444, 2024-09-16T13:00:01.453 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16303 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754344+0000", 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754344+0000", 2024-09-16T13:00:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754344+0000", 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754345+0000", 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0899999999999999e-06 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754349+0000", 2024-09-16T13:00:01.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9790000000000004e-06 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754356+0000", 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8279999999999997e-06 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766418+0000", 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012062226000000001 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766488+0000", 2024-09-16T13:00:01.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9438999999999997e-05 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766564+0000", 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6514999999999995e-05 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16304 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754361+0000", 2024-09-16T13:00:01.457 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.607332166999999, 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012417071999999999, 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16304 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754361+0000", 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754361+0000", 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754362+0000", 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754363+0000", 2024-09-16T13:00:01.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.28e-06 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754368+0000", 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.605e-06 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754371+0000", 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9239999999999999e-06 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766605+0000", 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012234282000000001 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766701+0000", 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6175999999999995e-05 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766779+0000", 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7454999999999996e-05 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.461 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:42521 4.8 4:15657af1:::rbd_data.11b52dde2e.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:01.158529+0000", 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20316474500000001, 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035255728, 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 42521 2024-09-16T13:00:01.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158529+0000", 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158529+0000", 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158530+0000", 2024-09-16T13:00:01.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.108e-06 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158536+0000", 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1290000000000001e-06 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158554+0000", 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7833e-05 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:01.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158561+0000", 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5159999999999997e-06 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158700+0000", 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138071 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158833+0000", 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013376499999999999 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:01.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.169342+0000", 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010508999999999999 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.193701+0000", 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024358956000000001 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.193712+0000", 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1093e-05 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.193754+0000", 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1995e-05 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.193785+0000", 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0261999999999999e-05 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:01.467 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:01.469 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:01.470 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:01.462+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:01.470 INFO:teuthology.orchestra.run.smithi042.stdout:2751 writesame 0xe8ffbcb thru 0xe90628e data_size 0x2(0x66c4 bytes) 2024-09-16T13:00:01.470 INFO:teuthology.orchestra.run.smithi042.stdout:2752 read 0xda5fb0a thru 0xda6bb9a (0xc091 bytes) 2024-09-16T13:00:01.470 INFO:teuthology.orchestra.run.smithi042.stdout:2754 punch from 0x4e4d09e to 0x4e51866, (0x47c8 bytes) 2024-09-16T13:00:01.470 INFO:teuthology.orchestra.run.smithi042.stdout:2755 writesame 0x3b3f5db thru 0x3b44ad4 data_size 0x2(0x54fa bytes) 2024-09-16T13:00:01.470 INFO:teuthology.orchestra.run.smithi042.stdout:2756 read 0x5d5f3eb thru 0x5d6b68c (0xc2a2 bytes) 2024-09-16T13:00:01.471 INFO:teuthology.orchestra.run.smithi042.stdout:2757 writesame 0x919ff96 thru 0x91ab4bf data_size 0x1(0xb52a bytes) 2024-09-16T13:00:01.471 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.471 INFO:teuthology.orchestra.run.smithi042.stdout:2759 trunc from 0xe90628f to 0xea1c4a8 2024-09-16T13:00:01.471 INFO:teuthology.orchestra.run.smithi042.stdout:2760 writesame 0x28233f3 thru 0x2831b17 data_size 0x1(0xe725 bytes) 2024-09-16T13:00:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:2761 write 0xebf4b7c thru 0xebf55a1 (0xa26 bytes) 2024-09-16T13:00:01.512 INFO:teuthology.orchestra.run.smithi042.stdout:2765 read 0x8394949 thru 0x83a432b (0xf9e3 bytes) 2024-09-16T13:00:01.513 INFO:teuthology.orchestra.run.smithi042.stdout:2766 write 0x6507c4a thru 0x6512297 (0xa64e bytes) 2024-09-16T13:00:01.516 INFO:teuthology.orchestra.run.smithi042.stdout:2767 write 0x7d5f1f1 thru 0x7d62c3f (0x3a4f bytes) 2024-09-16T13:00:01.518 INFO:teuthology.orchestra.run.smithi042.stdout:2768 write 0xc5ad733 thru 0xc5ae762 (0x1030 bytes) 2024-09-16T13:00:01.522 INFO:teuthology.orchestra.run.smithi042.stdout:2771 punch from 0x3605bdd to 0x36121bc, (0xc5df bytes) 2024-09-16T13:00:01.522 INFO:teuthology.orchestra.run.smithi042.stdout:2772 punch from 0x256c495 to 0x256e622, (0x218d bytes) 2024-09-16T13:00:01.522 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.522 INFO:teuthology.orchestra.run.smithi042.stdout:2774 writesame 0x39222ae thru 0x392c8cb data_size 0xb(0xa61e bytes) 2024-09-16T13:00:01.527 INFO:teuthology.orchestra.run.smithi042.stdout:2775 writesame 0x507b8bb thru 0x507c03e data_size 0x4(0x784 bytes) 2024-09-16T13:00:01.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:01.529 INFO:teuthology.orchestra.run.smithi042.stdout:2777 punch from 0x5315011 to 0x5317133, (0x2122 bytes) 2024-09-16T13:00:01.530 INFO:teuthology.orchestra.run.smithi042.stdout:2778 write 0x89413fe thru 0x894d5c6 (0xc1c9 bytes) 2024-09-16T13:00:01.532 INFO:teuthology.orchestra.run.smithi042.stdout:2779 write 0x6d136ea thru 0x6d21abe (0xe3d5 bytes) 2024-09-16T13:00:01.535 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.535 INFO:teuthology.orchestra.run.smithi042.stdout:2782 writesame 0x50cfcd1 thru 0x50d08d1 data_size 0x7(0xc01 bytes) 2024-09-16T13:00:01.538 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.550 INFO:teuthology.orchestra.run.smithi042.stdout:2784 trunc from 0xebf55a2 to 0x880d0d2 2024-09-16T13:00:01.569 INFO:teuthology.orchestra.run.smithi042.stdout:2785 trunc from 0x880d0d2 to 0xb706336 2024-09-16T13:00:01.570 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:01.562+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:01.579 INFO:teuthology.orchestra.run.smithi042.stdout:2786 read 0x8d0d126 thru 0x8d13c7c (0x6b57 bytes) 2024-09-16T13:00:01.580 INFO:teuthology.orchestra.run.smithi042.stdout:2788 punch from 0x4526928 to 0x45289cc, (0x20a4 bytes) 2024-09-16T13:00:01.580 INFO:teuthology.orchestra.run.smithi042.stdout:2790 writesame 0x8f925ff thru 0x8f97fac data_size 0xd(0x59ae bytes) 2024-09-16T13:00:01.583 INFO:teuthology.orchestra.run.smithi042.stdout:2791 write 0x3e08a7e thru 0x3e16e77 (0xe3fa bytes) 2024-09-16T13:00:01.587 INFO:teuthology.orchestra.run.smithi042.stdout:2793 write 0x73f04e1 thru 0x73f9b57 (0x9677 bytes) 2024-09-16T13:00:01.590 INFO:teuthology.orchestra.run.smithi042.stdout:2794 punch from 0x3c9135c to 0x3c95aab, (0x474f bytes) 2024-09-16T13:00:01.590 INFO:teuthology.orchestra.run.smithi042.stdout:2796 write 0xb64ecb3 thru 0xb654601 (0x594f bytes) 2024-09-16T13:00:01.593 INFO:teuthology.orchestra.run.smithi042.stdout:2797 read 0x78e6e7 thru 0x78eea1 (0x7bb bytes) 2024-09-16T13:00:01.594 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.594 INFO:teuthology.orchestra.run.smithi042.stdout:2800 punch from 0x3f745c0 to 0x3f82139, (0xdb79 bytes) 2024-09-16T13:00:01.595 INFO:teuthology.orchestra.run.smithi042.stdout:2801 read 0x7b24d1c thru 0x7b2edbf (0xa0a4 bytes) 2024-09-16T13:00:01.613 INFO:teuthology.orchestra.run.smithi042.stdout:2803 trunc from 0xb706336 to 0x269f13 2024-09-16T13:00:01.618 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:01.618 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:01.619 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:01.619 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:01.629 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:01.649 INFO:teuthology.orchestra.run.smithi042.stdout:2804 read 0x208c3a thru 0x20ef2d (0x62f4 bytes) 2024-09-16T13:00:01.670 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:01.666+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:01.671 INFO:teuthology.orchestra.run.smithi042.stdout:2805 trunc from 0x269f13 to 0xc01c9f4 2024-09-16T13:00:01.680 INFO:teuthology.orchestra.run.smithi042.stdout:2807 read 0x3ff2954 thru 0x3ff7110 (0x47bd bytes) 2024-09-16T13:00:01.681 INFO:teuthology.orchestra.run.smithi042.stdout:2808 writesame 0x8af3e41 thru 0x8afc956 data_size 0x13(0x8b16 bytes) 2024-09-16T13:00:01.683 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:01.686 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.686 INFO:teuthology.orchestra.run.smithi042.stdout:2810 read 0x2309823 thru 0x2319619 (0xfdf7 bytes) 2024-09-16T13:00:01.686 INFO:teuthology.orchestra.run.smithi042.stdout:2811 read 0xbd6d36a thru 0xbd6f1c7 (0x1e5e bytes) 2024-09-16T13:00:01.704 INFO:teuthology.orchestra.run.smithi042.stdout:2814 writesame 0xed9717d thru 0xeda35be data_size 0x1(0xc442 bytes) 2024-09-16T13:00:01.704 INFO:teuthology.orchestra.run.smithi042.stdout:2815 read 0x139f55d thru 0x13acae8 (0xd58c bytes) 2024-09-16T13:00:01.704 INFO:teuthology.orchestra.run.smithi042.stdout:2816 write 0x60f16f5 thru 0x60ff90e (0xe21a bytes) 2024-09-16T13:00:01.710 INFO:teuthology.orchestra.run.smithi042.stdout:2818 punch from 0xbda186e to 0xbdae7c4, (0xcf56 bytes) 2024-09-16T13:00:01.711 INFO:teuthology.orchestra.run.smithi042.stdout:2819 read 0x149d9aa thru 0x14a381a (0x5e71 bytes) 2024-09-16T13:00:01.711 INFO:teuthology.orchestra.run.smithi042.stdout:2820 write 0x31999b thru 0x31c284 (0x28ea bytes) 2024-09-16T13:00:01.712 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:01.713 INFO:teuthology.orchestra.run.smithi042.stdout:2821 punch from 0x2c8a69e to 0x2c961d6, (0xbb38 bytes) 2024-09-16T13:00:01.713 INFO:teuthology.orchestra.run.smithi042.stdout:2823 write 0x9c0113e thru 0x9c076e2 (0x65a5 bytes) 2024-09-16T13:00:01.718 INFO:teuthology.orchestra.run.smithi042.stdout:2824 write 0x70ce203 thru 0x70db0b9 (0xceb7 bytes) 2024-09-16T13:00:01.721 INFO:teuthology.orchestra.run.smithi042.stdout:2825 read 0x7036678 thru 0x704634a (0xfcd3 bytes) 2024-09-16T13:00:01.722 INFO:teuthology.orchestra.run.smithi042.stdout:2826 writesame 0x8c29fd1 thru 0x8c35d80 data_size 0xa(0xbdb0 bytes) 2024-09-16T13:00:01.724 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:01.725 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:01.730 INFO:teuthology.orchestra.run.smithi042.stdout:2828 punch from 0xc821937 to 0xc82cdae, (0xb477 bytes) 2024-09-16T13:00:01.730 INFO:teuthology.orchestra.run.smithi042.stdout:2829 writesame 0x619ccd6 thru 0x61a7bc3 data_size 0x1(0xaeee bytes) 2024-09-16T13:00:01.734 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:01.739 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:01.756 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:01.759 INFO:teuthology.orchestra.run.smithi042.stdout:2830 write 0x966cbf8 thru 0x9671dd7 (0x51e0 bytes) 2024-09-16T13:00:01.765 INFO:teuthology.orchestra.run.smithi042.stdout:2831 punch from 0x7102bc4 to 0x7112b10, (0xff4c bytes) 2024-09-16T13:00:01.765 INFO:teuthology.orchestra.run.smithi042.stdout:2832 write 0xc53179c thru 0xc536ec8 (0x572d bytes) 2024-09-16T13:00:01.768 INFO:teuthology.orchestra.run.smithi042.stdout:2833 writesame 0xcb94053 thru 0xcb98478 data_size 0x1a(0x4426 bytes) 2024-09-16T13:00:01.774 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:01.770+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:01.776 INFO:teuthology.orchestra.run.smithi042.stdout:2834 read 0x2a271c2 thru 0x2a319e0 (0xa81f bytes) 2024-09-16T13:00:01.776 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.778 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:00:01.779 INFO:teuthology.orchestra.run.smithi042.stdout:2839 trunc from 0xeda35bf to 0xd037f7a 2024-09-16T13:00:01.792 INFO:teuthology.orchestra.run.smithi042.stdout:2840 write 0x81bf514 thru 0x81cbb40 (0xc62d bytes) 2024-09-16T13:00:01.798 INFO:teuthology.orchestra.run.smithi042.stdout:2841 read 0xb15bfb2 thru 0xb1647d3 (0x8822 bytes) 2024-09-16T13:00:01.798 INFO:teuthology.orchestra.run.smithi042.stdout:2842 write 0x7337496 thru 0x73463b0 (0xef1b bytes) 2024-09-16T13:00:01.801 INFO:teuthology.orchestra.run.smithi042.stdout:2844 write 0xb7060ad thru 0xb70aa0b (0x495f bytes) 2024-09-16T13:00:01.803 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:01.820 INFO:teuthology.orchestra.run.smithi042.stdout:2848 write 0xdcb4210 thru 0xdcbd016 (0x8e07 bytes) 2024-09-16T13:00:01.825 INFO:teuthology.orchestra.run.smithi042.stdout:2849 read 0xc13687e thru 0xc1449c6 (0xe149 bytes) 2024-09-16T13:00:01.825 INFO:teuthology.orchestra.run.smithi042.stdout:2850 read 0x8abb949 thru 0x8ac3838 (0x7ef0 bytes) 2024-09-16T13:00:01.826 INFO:teuthology.orchestra.run.smithi042.stdout:2851 punch from 0x717563 to 0x71d29d, (0x5d3a bytes) 2024-09-16T13:00:01.826 INFO:teuthology.orchestra.run.smithi042.stdout:2852 read 0x768a5c8 thru 0x769a56b (0xffa4 bytes) 2024-09-16T13:00:01.826 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:01.826 INFO:teuthology.orchestra.run.smithi042.stdout:2854 write 0x8e52242 thru 0x8e5e66a (0xc429 bytes) 2024-09-16T13:00:01.827 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:01.829 INFO:teuthology.orchestra.run.smithi042.stdout:2855 read 0x269e363 thru 0x26a42f7 (0x5f95 bytes) 2024-09-16T13:00:01.830 INFO:teuthology.orchestra.run.smithi042.stdout:2856 writesame 0xb3a2a2e thru 0xb3adf6c data_size 0x1(0xb53f bytes) 2024-09-16T13:00:01.852 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:00:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:01.871 INFO:teuthology.orchestra.run.smithi042.stdout:2857 writesame 0x49304c9 thru 0x4937941 data_size 0x3(0x7479 bytes) 2024-09-16T13:00:01.872 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:01.863+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:01.877 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:01.877 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:01.883 INFO:teuthology.orchestra.run.smithi042.stdout:2858 punch from 0xcee754c to 0xcef6f81, (0xfa35 bytes) 2024-09-16T13:00:01.883 INFO:teuthology.orchestra.run.smithi042.stdout:2859 read 0x340e3a4 thru 0x3419430 (0xb08d bytes) 2024-09-16T13:00:01.886 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:01.908 INFO:teuthology.orchestra.run.smithi042.stdout:2863 trunc from 0xdcbd017 to 0x1b73145 2024-09-16T13:00:01.934 INFO:teuthology.orchestra.run.smithi042.stdout:2864 read 0x57e65c thru 0x5882a3 (0x9c48 bytes) 2024-09-16T13:00:01.934 INFO:teuthology.orchestra.run.smithi042.stdout:2865 punch from 0x1197726 to 0x11a7375, (0xfc4f bytes) 2024-09-16T13:00:01.947 INFO:teuthology.orchestra.run.smithi042.stdout:2866 write 0x4df94a6 thru 0x4e0467b (0xb1d6 bytes) 2024-09-16T13:00:01.952 INFO:teuthology.orchestra.run.smithi042.stdout:2867 read 0x256d44a thru 0x256e71f (0x12d6 bytes) 2024-09-16T13:00:01.956 INFO:teuthology.orchestra.run.smithi042.stdout:2868 trunc from 0x4e0467c to 0x40ddeac 2024-09-16T13:00:01.967 INFO:teuthology.orchestra.run.smithi042.stdout:2870 read 0x33d75d1 thru 0x33e42c9 (0xccf9 bytes) 2024-09-16T13:00:01.973 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:01.966+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:01.989 INFO:teuthology.orchestra.run.smithi042.stdout:2871 write 0x8c20ef3 thru 0x8c21ce4 (0xdf2 bytes) 2024-09-16T13:00:01.990 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:01.990 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:01.996 INFO:teuthology.orchestra.run.smithi042.stdout:2872 trunc from 0x8c21ce5 to 0x801efb0 2024-09-16T13:00:02.000 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:02.011 INFO:teuthology.orchestra.run.smithi042.stdout:2875 read 0x2eb0fcb thru 0x2eb32de (0x2314 bytes) 2024-09-16T13:00:02.011 INFO:teuthology.orchestra.run.smithi042.stdout:2876 writesame 0x644663e thru 0x644fdaa data_size 0x1(0x976d bytes) 2024-09-16T13:00:02.046 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.069 INFO:teuthology.orchestra.run.smithi042.stdout:2879 write 0xdf3a3d6 thru 0xdf43049 (0x8c74 bytes) 2024-09-16T13:00:02.074 INFO:teuthology.orchestra.run.smithi042.stdout:2880 read 0x6a47b84 thru 0x6a5685c (0xecd9 bytes) 2024-09-16T13:00:02.074 INFO:teuthology.orchestra.run.smithi042.stdout:2881 punch from 0x1b8b7ca to 0x1b92c7c, (0x74b2 bytes) 2024-09-16T13:00:02.074 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.074 INFO:teuthology.orchestra.run.smithi042.stdout:2883 punch from 0x72757d5 to 0x7283621, (0xde4c bytes) 2024-09-16T13:00:02.074 INFO:teuthology.orchestra.run.smithi042.stdout:2884 read 0x119ee74 thru 0x11a2134 (0x32c1 bytes) 2024-09-16T13:00:02.074 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:02.070+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:02.094 INFO:teuthology.orchestra.run.smithi042.stdout:2885 trunc from 0xdf4304a to 0x29c36d9 2024-09-16T13:00:02.114 INFO:teuthology.orchestra.run.smithi042.stdout:2886 write 0x129c999 thru 0x129d5d7 (0xc3f bytes) 2024-09-16T13:00:02.117 INFO:teuthology.orchestra.run.smithi042.stdout:2887 write 0x1a2f719 thru 0x1a34b34 (0x541c bytes) 2024-09-16T13:00:02.132 INFO:teuthology.orchestra.run.smithi042.stdout:2888 write 0x3e81b87 thru 0x3e8e71a (0xcb94 bytes) 2024-09-16T13:00:02.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:02.151 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:02.157 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T13:00:02.161 INFO:teuthology.orchestra.run.smithi042.stdout:2889 write 0x7e741c3 thru 0x7e77c0c (0x3a4a bytes) 2024-09-16T13:00:02.162 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:02.166 INFO:teuthology.orchestra.run.smithi042.stdout:2891 writesame 0x67f56bb thru 0x6800d29 data_size 0x1(0xb66f bytes) 2024-09-16T13:00:02.176 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:02.167+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:02.196 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.196 INFO:teuthology.orchestra.run.smithi042.stdout:2893 read 0x62d954e thru 0x62de800 (0x52b3 bytes) 2024-09-16T13:00:02.212 INFO:teuthology.orchestra.run.smithi042.stdout:2894 write 0xabca7f4 thru 0xabd00f1 (0x58fe bytes) 2024-09-16T13:00:02.215 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.215 INFO:teuthology.orchestra.run.smithi042.stdout:2896 read 0x73fdb6 thru 0x74b762 (0xb9ad bytes) 2024-09-16T13:00:02.219 INFO:teuthology.orchestra.run.smithi042.stdout:2897 writesame 0xabac598 thru 0xabad57e data_size 0x17(0xfe7 bytes) 2024-09-16T13:00:02.220 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T13:00:02.220 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T13:00:02.220 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-16T13:00:02.220 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T13:00:02.220 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T13:00:02.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-16T13:00:02.223 INFO:teuthology.orchestra.run.smithi042.stdout:2898 write 0x6884abc thru 0x688f5ef (0xab34 bytes) 2024-09-16T13:00:02.225 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.225 INFO:teuthology.orchestra.run.smithi042.stdout:2903 punch from 0x5411889 to 0x54209d6, (0xf14d bytes) 2024-09-16T13:00:02.225 INFO:teuthology.orchestra.run.smithi042.stdout:2906 punch from 0xddfe76 to 0xde6dea, (0x6f74 bytes) 2024-09-16T13:00:02.226 INFO:teuthology.orchestra.run.smithi042.stdout:2907 write 0x8065513 thru 0x8073578 (0xe066 bytes) 2024-09-16T13:00:02.234 INFO:teuthology.orchestra.run.smithi042.stdout:2909 trunc from 0xabd00f2 to 0x879009c 2024-09-16T13:00:02.245 INFO:teuthology.orchestra.run.smithi042.stdout:2910 read 0x54cf5a4 thru 0x54ddd29 (0xe786 bytes) 2024-09-16T13:00:02.258 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:02.258 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:02.263 INFO:teuthology.orchestra.run.smithi042.stdout:2911 write 0xd12b71d thru 0xd13393b (0x821f bytes) 2024-09-16T13:00:02.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:02.268 INFO:teuthology.orchestra.run.smithi042.stdout:2914 read 0x9064b4f thru 0x906bc5e (0x7110 bytes) 2024-09-16T13:00:02.268 INFO:teuthology.orchestra.run.smithi042.stdout:2915 write 0x4cca06 thru 0x4d203d (0x5638 bytes) 2024-09-16T13:00:02.270 INFO:teuthology.orchestra.run.smithi042.stdout:2916 writesame 0x3b75827 thru 0x3b791a0 data_size 0x7(0x397a bytes) 2024-09-16T13:00:02.274 INFO:teuthology.orchestra.run.smithi042.stdout:2917 punch from 0x7251d03 to 0x725774d, (0x5a4a bytes) 2024-09-16T13:00:02.278 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:02.270+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:02.287 INFO:teuthology.orchestra.run.smithi042.stdout:2918 writesame 0xecbcebf thru 0xecc92db data_size 0x5(0xc41d bytes) 2024-09-16T13:00:02.289 INFO:teuthology.orchestra.run.smithi042.stdout:2919 punch from 0xf3bada to 0xf47876, (0xbd9c bytes) 2024-09-16T13:00:02.289 INFO:teuthology.orchestra.run.smithi042.stdout:2920 read 0xbfbda67 thru 0xbfc5fe4 (0x857e bytes) 2024-09-16T13:00:02.289 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.289 INFO:teuthology.orchestra.run.smithi042.stdout:2922 write 0xe225766 thru 0xe226130 (0x9cb bytes) 2024-09-16T13:00:02.291 INFO:teuthology.orchestra.run.smithi042.stdout:2923 write 0xde0a33 thru 0xdea875 (0x9e43 bytes) 2024-09-16T13:00:02.294 INFO:teuthology.orchestra.run.smithi042.stdout:2925 writesame 0x19d5fc0 thru 0x19e30b8 data_size 0x1(0xd0f9 bytes) 2024-09-16T13:00:02.326 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.327 INFO:teuthology.orchestra.run.smithi042.stdout:2928 read 0x9b802b3 thru 0x9b8fe3d (0xfb8b bytes) 2024-09-16T13:00:02.327 INFO:teuthology.orchestra.run.smithi042.stdout:2929 writesame 0x5f558ac thru 0x5f59cea data_size 0x1(0x443f bytes) 2024-09-16T13:00:02.327 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:02.328 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T13:00:02.328 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:02.346 INFO:teuthology.orchestra.run.smithi042.stdout:2930 trunc from 0xecc92dc to 0xbe647de 2024-09-16T13:00:02.360 INFO:teuthology.orchestra.run.smithi042.stdout:2931 punch from 0xa0ee758 to 0xa0f9a30, (0xb2d8 bytes) 2024-09-16T13:00:02.360 INFO:teuthology.orchestra.run.smithi042.stdout:2932 writesame 0xca5cdc thru 0xcb4faa data_size 0x1(0xf2cf bytes) 2024-09-16T13:00:02.376 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:02.376 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:02.379 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:02.371+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:02.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:02.394 INFO:teuthology.orchestra.run.smithi042.stdout:2933 writesame 0x2f06cbe thru 0x2f0b0df data_size 0x9(0x4422 bytes) 2024-09-16T13:00:02.411 INFO:teuthology.orchestra.run.smithi042.stdout:2934 writesame 0xea60bc1 thru 0xea63010 data_size 0x1c(0x2450 bytes) 2024-09-16T13:00:02.430 INFO:teuthology.orchestra.run.smithi042.stdout:2935 trunc from 0xea63011 to 0x4a9acb1 2024-09-16T13:00:02.448 INFO:teuthology.orchestra.run.smithi042.stdout:2936 trunc from 0x4a9acb1 to 0x7896078 2024-09-16T13:00:02.457 INFO:teuthology.orchestra.run.smithi042.stdout:2938 punch from 0x4829e99 to 0x48319e9, (0x7b50 bytes) 2024-09-16T13:00:02.457 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.479 INFO:teuthology.orchestra.run.smithi042.stdout:2942 write 0xee34b1e thru 0xee3e241 (0x9724 bytes) 2024-09-16T13:00:02.479 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:02.471+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:02.482 INFO:teuthology.orchestra.run.smithi042.stdout:2944 write 0x749db42 thru 0x74a1cf1 (0x41b0 bytes) 2024-09-16T13:00:02.496 INFO:teuthology.orchestra.run.smithi042.stdout:2947 trunc from 0xee3e242 to 0x920e716 2024-09-16T13:00:02.500 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:02.500 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:02.510 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:02.512 INFO:teuthology.orchestra.run.smithi042.stdout:2949 punch from 0x415026b to 0x41507a1, (0x536 bytes) 2024-09-16T13:00:02.512 INFO:teuthology.orchestra.run.smithi042.stdout:2951 read 0x350d1ab thru 0x3515bac (0x8a02 bytes) 2024-09-16T13:00:02.513 INFO:teuthology.orchestra.run.smithi042.stdout:2952 write 0x76304a thru 0x76b7b0 (0x8767 bytes) 2024-09-16T13:00:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:2953 read 0x91bce04 thru 0x91c5678 (0x8875 bytes) 2024-09-16T13:00:02.531 INFO:teuthology.orchestra.run.smithi042.stdout:2954 write 0xb314598 thru 0xb31f3d3 (0xae3c bytes) 2024-09-16T13:00:02.550 INFO:teuthology.orchestra.run.smithi042.stdout:2955 write 0xdfe0dc9 thru 0xdfe4956 (0x3b8e bytes) 2024-09-16T13:00:02.556 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.578 INFO:teuthology.orchestra.run.smithi042.stdout:2957 trunc from 0xdfe4957 to 0xe73578 2024-09-16T13:00:02.580 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:02.574+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:02.625 INFO:teuthology.orchestra.run.smithi042.stdout:2959 write 0x24644cc thru 0x246891d (0x4452 bytes) 2024-09-16T13:00:02.625 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:02.625 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:02.632 INFO:teuthology.orchestra.run.smithi042.stdout:2961 punch from 0x23630ab to 0x2363c8b, (0xbe0 bytes) 2024-09-16T13:00:02.632 INFO:teuthology.orchestra.run.smithi042.stdout:2964 punch from 0x1d21e92 to 0x1d29065, (0x71d3 bytes) 2024-09-16T13:00:02.632 INFO:teuthology.orchestra.run.smithi042.stdout:2965 punch from 0x651312 to 0x6518eb, (0x5d9 bytes) 2024-09-16T13:00:02.632 INFO:teuthology.orchestra.run.smithi042.stdout:2966 write 0x1fa4b5d thru 0x1faf727 (0xabcb bytes) 2024-09-16T13:00:02.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:02.638 INFO:teuthology.orchestra.run.smithi042.stdout:2969 punch from 0x2198612 to 0x219acee, (0x26dc bytes) 2024-09-16T13:00:02.638 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.653 INFO:teuthology.orchestra.run.smithi042.stdout:2973 write 0x4c22bca thru 0x4c3158a (0xe9c1 bytes) 2024-09-16T13:00:02.664 INFO:teuthology.orchestra.run.smithi042.stdout:2974 trunc from 0x4c3158b to 0x1a0d8c8 2024-09-16T13:00:02.681 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:02.671+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:02.701 INFO:teuthology.orchestra.run.smithi042.stdout:2975 write 0x6a37c34 thru 0x6a4697b (0xed48 bytes) 2024-09-16T13:00:02.707 INFO:teuthology.orchestra.run.smithi042.stdout:2978 read 0xed3744 thru 0xed3aa0 (0x35d bytes) 2024-09-16T13:00:02.730 INFO:teuthology.orchestra.run.smithi042.stdout:2979 write 0xd4b79c5 thru 0xd4bab54 (0x3190 bytes) 2024-09-16T13:00:02.736 INFO:teuthology.orchestra.run.smithi042.stdout:2980 write 0x67e9bf6 thru 0x67f6ead (0xd2b8 bytes) 2024-09-16T13:00:02.738 INFO:teuthology.orchestra.run.smithi042.stdout:2982 writesame 0x10414d2 thru 0x104e980 data_size 0x1(0xd4af bytes) 2024-09-16T13:00:02.749 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:02.749 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:02.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:02.778 INFO:teuthology.orchestra.run.smithi042.stdout:2985 write 0xb4bfda0 thru 0xb4cfa8b (0xfcec bytes) 2024-09-16T13:00:02.780 INFO:teuthology.orchestra.run.smithi042.stdout:2987 read 0x2a7dccb thru 0x2a8d387 (0xf6bd bytes) 2024-09-16T13:00:02.780 INFO:teuthology.orchestra.run.smithi042.stdout:2990 write 0x1e058ba thru 0x1e11c75 (0xc3bc bytes) 2024-09-16T13:00:02.781 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:02.775+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:02.786 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.786 INFO:teuthology.orchestra.run.smithi042.stdout:2993 read 0x4a01c45 thru 0x4a080f8 (0x64b4 bytes) 2024-09-16T13:00:02.786 INFO:teuthology.orchestra.run.smithi042.stdout:2994 write 0x6debe82 thru 0x6df65b3 (0xa732 bytes) 2024-09-16T13:00:02.802 INFO:teuthology.orchestra.run.smithi042.stdout:2995 writesame 0xe6cf4c6 thru 0xe6dc9c1 data_size 0x56(0xd4fc bytes) 2024-09-16T13:00:02.805 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.805 INFO:teuthology.orchestra.run.smithi042.stdout:2998 punch from 0x56d79f5 to 0x56d9a6c, (0x2077 bytes) 2024-09-16T13:00:02.808 INFO:teuthology.orchestra.run.smithi042.stdout:2999 trunc from 0xe6dc9c2 to 0xd2581fd 2024-09-16T13:00:02.821 INFO:teuthology.orchestra.run.smithi042.stdout:3000 read 0x13179a7 thru 0x1324caf (0xd309 bytes) 2024-09-16T13:00:02.821 INFO:teuthology.orchestra.run.smithi042.stdout:3001 write 0x77ac7d4 thru 0x77accc9 (0x4f6 bytes) 2024-09-16T13:00:02.825 INFO:teuthology.orchestra.run.smithi042.stdout:3003 read 0x7b095c6 thru 0x7b0d7ab (0x41e6 bytes) 2024-09-16T13:00:02.825 INFO:teuthology.orchestra.run.smithi042.stdout:3004 write 0xa72889a thru 0xa72a4ef (0x1c56 bytes) 2024-09-16T13:00:02.829 INFO:teuthology.orchestra.run.smithi042.stdout:3005 write 0x87d9864 thru 0x87e8207 (0xe9a4 bytes) 2024-09-16T13:00:02.831 INFO:teuthology.orchestra.run.smithi042.stdout:3006 writesame 0x4328264 thru 0x4337bcf data_size 0x66(0xf96c bytes) 2024-09-16T13:00:02.835 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.835 INFO:teuthology.orchestra.run.smithi042.stdout:3010 punch from 0xa286556 to 0xa28fec3, (0x996d bytes) 2024-09-16T13:00:02.854 INFO:teuthology.orchestra.run.smithi042.stdout:3011 trunc from 0xd2581fd to 0x1bb50bc 2024-09-16T13:00:02.882 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:02.875+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:02.888 INFO:teuthology.orchestra.run.smithi042.stdout:3013 writesame 0x3cb7de7 thru 0x3cc721a data_size 0x2(0xf434 bytes) 2024-09-16T13:00:02.892 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.892 INFO:teuthology.orchestra.run.smithi042.stdout:3017 read 0x1b8c65b thru 0x1b8e7fd (0x21a3 bytes) 2024-09-16T13:00:02.894 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:02.894 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:02.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:02.907 INFO:teuthology.orchestra.run.smithi042.stdout:3018 write 0x6ee0c97 thru 0x6eece11 (0xc17b bytes) 2024-09-16T13:00:02.912 INFO:teuthology.orchestra.run.smithi042.stdout:3019 punch from 0x4efd89e to 0x4f033b7, (0x5b19 bytes) 2024-09-16T13:00:02.912 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:02.912 INFO:teuthology.orchestra.run.smithi042.stdout:3021 read 0x3057e7a thru 0x305b7f1 (0x3978 bytes) 2024-09-16T13:00:02.912 INFO:teuthology.orchestra.run.smithi042.stdout:3022 punch from 0xdfc043 to 0xe0864d, (0xc60a bytes) 2024-09-16T13:00:02.912 INFO:teuthology.orchestra.run.smithi042.stdout:3024 read 0x8dcbb0 thru 0x8e9555 (0xc9a6 bytes) 2024-09-16T13:00:02.913 INFO:teuthology.orchestra.run.smithi042.stdout:3025 read 0x3abf836 thru 0x3ac5664 (0x5e2f bytes) 2024-09-16T13:00:02.914 INFO:teuthology.orchestra.run.smithi042.stdout:3026 read 0x53ecadf thru 0x53f0a0e (0x3f30 bytes) 2024-09-16T13:00:02.914 INFO:teuthology.orchestra.run.smithi042.stdout:3027 punch from 0x2c93093 to 0x2ca1f22, (0xee8f bytes) 2024-09-16T13:00:02.914 INFO:teuthology.orchestra.run.smithi042.stdout:3028 punch from 0x8dc731 to 0x8dc87d, (0x14c bytes) 2024-09-16T13:00:02.931 INFO:teuthology.orchestra.run.smithi042.stdout:3029 writesame 0xaefbdcd thru 0xaf0413e data_size 0x19(0x8372 bytes) 2024-09-16T13:00:02.934 INFO:teuthology.orchestra.run.smithi042.stdout:3030 read 0xaa6f60 thru 0xaaa15b (0x31fc bytes) 2024-09-16T13:00:02.951 INFO:teuthology.orchestra.run.smithi042.stdout:3031 writesame 0xe17c4ab thru 0xe17d9b8 data_size 0x7(0x150e bytes) 2024-09-16T13:00:02.953 INFO:teuthology.orchestra.run.smithi042.stdout:3032 writesame 0xc3bc539 thru 0xc3c326e data_size 0x7(0x6d36 bytes) 2024-09-16T13:00:02.961 INFO:teuthology.orchestra.run.smithi042.stdout:3033 read 0x8ac7c55 thru 0x8ad503f (0xd3eb bytes) 2024-09-16T13:00:02.962 INFO:teuthology.orchestra.run.smithi042.stdout:3034 punch from 0x8ed614a to 0x8edcd3e, (0x6bf4 bytes) 2024-09-16T13:00:02.962 INFO:teuthology.orchestra.run.smithi042.stdout:3035 read 0x63baafd thru 0x63c4b5f (0xa063 bytes) 2024-09-16T13:00:02.972 INFO:teuthology.orchestra.run.smithi042.stdout:3036 trunc from 0xe17d9b9 to 0x77a6d8c 2024-09-16T13:00:02.983 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:02.978+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:02.996 INFO:teuthology.orchestra.run.smithi042.stdout:3037 write 0x8b9890c thru 0x8ba45c4 (0xbcb9 bytes) 2024-09-16T13:00:02.999 INFO:teuthology.orchestra.run.smithi042.stdout:3038 punch from 0x24a4b2f to 0x24ad4f7, (0x89c8 bytes) 2024-09-16T13:00:02.999 INFO:teuthology.orchestra.run.smithi042.stdout:3041 read 0x5f57081 thru 0x5f5d188 (0x6108 bytes) 2024-09-16T13:00:02.999 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.013 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:03.013 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:03.018 INFO:teuthology.orchestra.run.smithi042.stdout:3044 write 0xce231a1 thru 0xce2daf9 (0xa959 bytes) 2024-09-16T13:00:03.023 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:03.025 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.025 INFO:teuthology.orchestra.run.smithi042.stdout:3046 read 0xcd4eb64 thru 0xcd576a7 (0x8b44 bytes) 2024-09-16T13:00:03.025 INFO:teuthology.orchestra.run.smithi042.stdout:3048 write 0x2238945 thru 0x223ea41 (0x60fd bytes) 2024-09-16T13:00:03.032 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.032 INFO:teuthology.orchestra.run.smithi042.stdout:3050 write 0xc139992 thru 0xc13fbfd (0x626c bytes) 2024-09-16T13:00:03.055 INFO:teuthology.orchestra.run.smithi042.stdout:3051 trunc from 0xce2dafa to 0xf0518a 2024-09-16T13:00:03.085 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:03.075+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:03.107 INFO:teuthology.orchestra.run.smithi042.stdout:3053 writesame 0xbbd40d7 thru 0xbbe3962 data_size 0x2(0xf88c bytes) 2024-09-16T13:00:03.107 INFO:teuthology.orchestra.run.smithi042.stdout:3054 punch from 0x3eee042 to 0x3ef56d4, (0x7692 bytes) 2024-09-16T13:00:03.107 INFO:teuthology.orchestra.run.smithi042.stdout:3055 write 0x33794bb thru 0x337d17c (0x3cc2 bytes) 2024-09-16T13:00:03.112 INFO:teuthology.orchestra.run.smithi042.stdout:3056 punch from 0xa30559b to 0xa30816e, (0x2bd3 bytes) 2024-09-16T13:00:03.126 INFO:teuthology.orchestra.run.smithi042.stdout:3058 trunc from 0xbbe3963 to 0x37cdb00 2024-09-16T13:00:03.134 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:03.134 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:03.139 INFO:teuthology.orchestra.run.smithi042.stdout:3059 write 0x20f2769 thru 0x2102400 (0xfc98 bytes) 2024-09-16T13:00:03.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:03.156 INFO:teuthology.orchestra.run.smithi042.stdout:3060 write 0x3c208ae thru 0x3c2a8c5 (0xa018 bytes) 2024-09-16T13:00:03.160 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.160 INFO:teuthology.orchestra.run.smithi042.stdout:3063 punch from 0x7d3dd8 to 0x7e3889, (0xfab1 bytes) 2024-09-16T13:00:03.160 INFO:teuthology.orchestra.run.smithi042.stdout:3064 read 0xab80df thru 0xac072e (0x8650 bytes) 2024-09-16T13:00:03.161 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.161 INFO:teuthology.orchestra.run.smithi042.stdout:3066 punch from 0x38efe5e to 0x38fcdbf, (0xcf61 bytes) 2024-09-16T13:00:03.161 INFO:teuthology.orchestra.run.smithi042.stdout:3068 writesame 0x1d4025c thru 0x1d4df79 data_size 0x53(0xdd1e bytes) 2024-09-16T13:00:03.167 INFO:teuthology.orchestra.run.smithi042.stdout:3071 punch from 0x136273 to 0x1414a6, (0xb233 bytes) 2024-09-16T13:00:03.186 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:03.179+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:03.199 INFO:teuthology.orchestra.run.smithi042.stdout:3072 write 0xe10e549 thru 0xe112864 (0x431c bytes) 2024-09-16T13:00:03.202 INFO:teuthology.orchestra.run.smithi042.stdout:3073 read 0x791aee1 thru 0x792517d (0xa29d bytes) 2024-09-16T13:00:03.203 INFO:teuthology.orchestra.run.smithi042.stdout:3075 read 0x8567e76 thru 0x856fb66 (0x7cf1 bytes) 2024-09-16T13:00:03.203 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.203 INFO:teuthology.orchestra.run.smithi042.stdout:3078 write 0x866c28 thru 0x8705f8 (0x99d1 bytes) 2024-09-16T13:00:03.207 INFO:teuthology.orchestra.run.smithi042.stdout:3079 read 0x9dd6da9 thru 0x9dd84f6 (0x174e bytes) 2024-09-16T13:00:03.207 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.207 INFO:teuthology.orchestra.run.smithi042.stdout:3081 write 0x5ac9a53 thru 0x5ad4d75 (0xb323 bytes) 2024-09-16T13:00:03.211 INFO:teuthology.orchestra.run.smithi042.stdout:3082 read 0x6e406a5 thru 0x6e46f00 (0x685c bytes) 2024-09-16T13:00:03.211 INFO:teuthology.orchestra.run.smithi042.stdout:3083 write 0xd4e18c9 thru 0xd4efa63 (0xe19b bytes) 2024-09-16T13:00:03.218 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.218 INFO:teuthology.orchestra.run.smithi042.stdout:3085 write 0x7c02093 thru 0x7c0c584 (0xa4f2 bytes) 2024-09-16T13:00:03.221 INFO:teuthology.orchestra.run.smithi042.stdout:3086 punch from 0xc601ddb to 0xc61187d, (0xfaa2 bytes) 2024-09-16T13:00:03.221 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.221 INFO:teuthology.orchestra.run.smithi042.stdout:3088 punch from 0x6658155 to 0x665b8b9, (0x3764 bytes) 2024-09-16T13:00:03.222 INFO:teuthology.orchestra.run.smithi042.stdout:3089 punch from 0x920b71a to 0x9211fb9, (0x689f bytes) 2024-09-16T13:00:03.238 INFO:teuthology.orchestra.run.smithi042.stdout:3091 trunc from 0xe112865 to 0x4252820 2024-09-16T13:00:03.254 INFO:teuthology.orchestra.run.smithi042.stdout:3093 read 0x29b3376 thru 0x29c1d35 (0xe9c0 bytes) 2024-09-16T13:00:03.254 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.254 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.254 INFO:teuthology.orchestra.run.smithi042.stdout:3097 punch from 0x25636e5 to 0x256f193, (0xbaae bytes) 2024-09-16T13:00:03.254 INFO:teuthology.orchestra.run.smithi042.stdout:3098 write 0x181bb7f thru 0x1821ea1 (0x6323 bytes) 2024-09-16T13:00:03.259 INFO:teuthology.orchestra.run.smithi042.stdout:3099 read 0x75c4a1 thru 0x75cf69 (0xac9 bytes) 2024-09-16T13:00:03.270 INFO:teuthology.orchestra.run.smithi042.stdout:3100 write 0x4cfa6d2 thru 0x4cfe99d (0x42cc bytes) 2024-09-16T13:00:03.275 INFO:teuthology.orchestra.run.smithi042.stdout:3101 read 0xe10c14 thru 0xe12594 (0x1981 bytes) 2024-09-16T13:00:03.276 INFO:teuthology.orchestra.run.smithi042.stdout:3102 read 0x2a4ed64 thru 0x2a54e68 (0x6105 bytes) 2024-09-16T13:00:03.276 INFO:teuthology.orchestra.run.smithi042.stdout:3103 read 0x1acc0e8 thru 0x1acf206 (0x311f bytes) 2024-09-16T13:00:03.278 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:03.278 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:03.287 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:03.282+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:03.290 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:03.302 INFO:teuthology.orchestra.run.smithi042.stdout:3109 writesame 0xdcf1b8b thru 0xdd00909 data_size 0x1(0xed7f bytes) 2024-09-16T13:00:03.302 INFO:teuthology.orchestra.run.smithi042.stdout:3111 write 0xd99a1fc thru 0xd99c754 (0x2559 bytes) 2024-09-16T13:00:03.304 INFO:teuthology.orchestra.run.smithi042.stdout:3112 punch from 0x9bed3b8 to 0x9bf2a42, (0x568a bytes) 2024-09-16T13:00:03.304 INFO:teuthology.orchestra.run.smithi042.stdout:3114 write 0x7b2d2cd thru 0x7b3839e (0xb0d2 bytes) 2024-09-16T13:00:03.307 INFO:teuthology.orchestra.run.smithi042.stdout:3115 read 0xbf1e22d thru 0xbf21630 (0x3404 bytes) 2024-09-16T13:00:03.308 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.308 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.308 INFO:teuthology.orchestra.run.smithi042.stdout:3118 punch from 0xb0077fd to 0xb0146d8, (0xcedb bytes) 2024-09-16T13:00:03.308 INFO:teuthology.orchestra.run.smithi042.stdout:3119 punch from 0xcb64a6d to 0xcb68b37, (0x40ca bytes) 2024-09-16T13:00:03.308 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.315 INFO:teuthology.orchestra.run.smithi042.stdout:3122 trunc from 0xdd0090a to 0x9ded5a6 2024-09-16T13:00:03.327 INFO:teuthology.orchestra.run.smithi042.stdout:3123 writesame 0x3c3e137 thru 0x3c496ed data_size 0x1(0xb5b7 bytes) 2024-09-16T13:00:03.365 INFO:teuthology.orchestra.run.smithi042.stdout:3125 read 0x7183007 thru 0x7190f2c (0xdf26 bytes) 2024-09-16T13:00:03.366 INFO:teuthology.orchestra.run.smithi042.stdout:3127 read 0x912b727 thru 0x912d1a3 (0x1a7d bytes) 2024-09-16T13:00:03.381 INFO:teuthology.orchestra.run.smithi042.stdout:3128 trunc from 0x9ded5a6 to 0x350e20c 2024-09-16T13:00:03.388 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:03.379+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:03.394 INFO:teuthology.orchestra.run.smithi042.stdout:3129 read 0x2fa1c4f thru 0x2fac8a6 (0xac58 bytes) 2024-09-16T13:00:03.394 INFO:teuthology.orchestra.run.smithi042.stdout:3130 punch from 0x15e072f to 0x15ea946, (0xa217 bytes) 2024-09-16T13:00:03.405 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:03.405 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:03.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:03.421 INFO:teuthology.orchestra.run.smithi042.stdout:3131 writesame 0xcae9fde thru 0xcaef6f1 data_size 0x2(0x5714 bytes) 2024-09-16T13:00:03.423 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.424 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.424 INFO:teuthology.orchestra.run.smithi042.stdout:3136 read 0xa2e426e thru 0xa2f2b3b (0xe8ce bytes) 2024-09-16T13:00:03.424 INFO:teuthology.orchestra.run.smithi042.stdout:3137 write 0xbf829d3 thru 0xbf8edcb (0xc3f9 bytes) 2024-09-16T13:00:03.426 INFO:teuthology.orchestra.run.smithi042.stdout:3140 read 0x3bfa147 thru 0x3c04256 (0xa110 bytes) 2024-09-16T13:00:03.445 INFO:teuthology.orchestra.run.smithi042.stdout:3143 trunc from 0xcaef6f2 to 0x397f993 2024-09-16T13:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout:3146 trunc from 0x397f993 to 0x442970 2024-09-16T13:00:03.488 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:03.482+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:3147 read 0x159486 thru 0x167cfe (0xe879 bytes) 2024-09-16T13:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout:3148 punch from 0x255b19 to 0x259370, (0x3857 bytes) 2024-09-16T13:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout:3149 write 0x7f34478 thru 0x7f43247 (0xedd0 bytes) 2024-09-16T13:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout:3151 read 0x4e24955 thru 0x4e2ed73 (0xa41f bytes) 2024-09-16T13:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout:3152 write 0x58227b thru 0x585904 (0x368a bytes) 2024-09-16T13:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout:3153 punch from 0x276b7fa to 0x277960a, (0xde10 bytes) 2024-09-16T13:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout:3154 read 0x2f906fd thru 0x2f9e0a1 (0xd9a5 bytes) 2024-09-16T13:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout:3156 write 0x95045c6 thru 0x95078e0 (0x331b bytes) 2024-09-16T13:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:03.549 INFO:teuthology.orchestra.run.smithi042.stdout:3157 read 0x2abb943 thru 0x2ac5462 (0x9b20 bytes) 2024-09-16T13:00:03.549 INFO:teuthology.orchestra.run.smithi042.stdout:3159 read 0x814dde4 thru 0x815a7e3 (0xca00 bytes) 2024-09-16T13:00:03.550 INFO:teuthology.orchestra.run.smithi042.stdout:3160 read 0x1a8216f thru 0x1a8eea8 (0xcd3a bytes) 2024-09-16T13:00:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:3161 trunc from 0x95078e1 to 0x14f4c4c 2024-09-16T13:00:03.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:03.610 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:03.603+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:03.611 INFO:teuthology.orchestra.run.smithi042.stdout:3163 write 0x9eba75c thru 0x9ec2385 (0x7c2a bytes) 2024-09-16T13:00:03.618 INFO:teuthology.orchestra.run.smithi042.stdout:3165 write 0x34bcf23 thru 0x34c1ed4 (0x4fb2 bytes) 2024-09-16T13:00:03.623 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.635 INFO:teuthology.orchestra.run.smithi042.stdout:3167 trunc from 0x9ec2386 to 0x48d0745 2024-09-16T13:00:03.650 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.650 INFO:teuthology.orchestra.run.smithi042.stdout:3171 read 0x23f4988 thru 0x23fb8eb (0x6f64 bytes) 2024-09-16T13:00:03.650 INFO:teuthology.orchestra.run.smithi042.stdout:3172 read 0x101d540 thru 0x1024d4f (0x7810 bytes) 2024-09-16T13:00:03.652 INFO:teuthology.orchestra.run.smithi042.stdout:3173 trunc from 0x48d0745 to 0x588d532 2024-09-16T13:00:03.678 INFO:teuthology.orchestra.run.smithi042.stdout:3174 trunc from 0x588d532 to 0xc7e35c0 2024-09-16T13:00:03.689 INFO:teuthology.orchestra.run.smithi042.stdout:3175 flatten 2024-09-16T13:00:03.690 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:03.682+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:03.700 INFO:teuthology.orchestra.run.smithi042.stdout:3176 read 0x5a1cf71 thru 0x5a26e6f (0x9eff bytes) 2024-09-16T13:00:03.700 INFO:teuthology.orchestra.run.smithi042.stdout:3177 read 0x482ca6f thru 0x48378a2 (0xae34 bytes) 2024-09-16T13:00:03.716 INFO:teuthology.orchestra.run.smithi042.stdout:3178 trunc from 0xc7e35c0 to 0x37e42b3 2024-09-16T13:00:03.717 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:03.717 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:03.727 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:03.729 INFO:teuthology.orchestra.run.smithi042.stdout:3180 read 0x5255c0 thru 0x531d16 (0xc757 bytes) 2024-09-16T13:00:03.742 INFO:teuthology.orchestra.run.smithi042.stdout:3182 write 0x4409e6b thru 0x44164c0 (0xc656 bytes) 2024-09-16T13:00:03.746 INFO:teuthology.orchestra.run.smithi042.stdout:3183 trunc from 0x44164c1 to 0x50eeccb 2024-09-16T13:00:03.757 INFO:teuthology.orchestra.run.smithi042.stdout:3184 punch from 0x1b01c20 to 0x1b0b398, (0x9778 bytes) 2024-09-16T13:00:03.757 INFO:teuthology.orchestra.run.smithi042.stdout:3185 punch from 0x11623d7 to 0x1171e03, (0xfa2c bytes) 2024-09-16T13:00:03.757 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.757 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.763 INFO:teuthology.orchestra.run.smithi042.stdout:3188 trunc from 0x50eeccb to 0x87015bc 2024-09-16T13:00:03.774 INFO:teuthology.orchestra.run.smithi042.stdout:3189 read 0x56f93fd thru 0x57007ce (0x73d2 bytes) 2024-09-16T13:00:03.774 INFO:teuthology.orchestra.run.smithi042.stdout:3190 write 0x8342ddd thru 0x835107f (0xe2a3 bytes) 2024-09-16T13:00:03.790 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:03.783+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:03.794 INFO:teuthology.orchestra.run.smithi042.stdout:3191 write 0x940dc1c thru 0x9411978 (0x3d5d bytes) 2024-09-16T13:00:03.800 INFO:teuthology.orchestra.run.smithi042.stdout:3193 write 0x3bdb187 thru 0x3be598c (0xa806 bytes) 2024-09-16T13:00:03.804 INFO:teuthology.orchestra.run.smithi042.stdout:3194 read 0x50c119c thru 0x50c5597 (0x43fc bytes) 2024-09-16T13:00:03.804 INFO:teuthology.orchestra.run.smithi042.stdout:3195 punch from 0x5aa5698 to 0x5ab1583, (0xbeeb bytes) 2024-09-16T13:00:03.821 INFO:teuthology.orchestra.run.smithi042.stdout:3197 writesame 0xc1dd7e2 thru 0xc1eadd5 data_size 0x2(0xd5f4 bytes) 2024-09-16T13:00:03.824 INFO:teuthology.orchestra.run.smithi042.stdout:3198 trunc from 0xc1eadd6 to 0xadf0408 2024-09-16T13:00:03.827 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:03.827 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:03.837 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:00:03.838 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.838 INFO:teuthology.orchestra.run.smithi042.stdout:3200 punch from 0x888684d to 0x8887f5e, (0x1711 bytes) 2024-09-16T13:00:03.838 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.838 INFO:teuthology.orchestra.run.smithi042.stdout:3202 punch from 0x7d4f33 to 0x7d83c1, (0x348e bytes) 2024-09-16T13:00:03.838 INFO:teuthology.orchestra.run.smithi042.stdout:3203 write 0x8b1efc9 thru 0x8b29099 (0xa0d1 bytes) 2024-09-16T13:00:03.842 INFO:teuthology.orchestra.run.smithi042.stdout:3205 write 0x90975b8 thru 0x90a61bb (0xec04 bytes) 2024-09-16T13:00:03.849 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.863 INFO:teuthology.orchestra.run.smithi042.stdout:3207 writesame 0xc631824 thru 0xc63a5f4 data_size 0x35(0x8dd1 bytes) 2024-09-16T13:00:03.866 INFO:teuthology.orchestra.run.smithi042.stdout:3208 trunc from 0xc63a5f5 to 0xdeae461 2024-09-16T13:00:03.872 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:03.879 INFO:teuthology.orchestra.run.smithi042.stdout:3209 read 0xba09e38 thru 0xba0bdef (0x1fb8 bytes) 2024-09-16T13:00:03.889 INFO:teuthology.orchestra.run.smithi042.stdout:3211 trunc from 0xdeae461 to 0x90ae083 2024-09-16T13:00:03.891 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:03.886+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:03.904 INFO:teuthology.orchestra.run.smithi042.stdout:3213 read 0x295b38a thru 0x2968250 (0xcec7 bytes) 2024-09-16T13:00:03.922 INFO:teuthology.orchestra.run.smithi042.stdout:3214 writesame 0xc8bb741 thru 0xc8c5bf4 data_size 0x1(0xa4b4 bytes) 2024-09-16T13:00:03.925 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.925 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:03.926 INFO:teuthology.orchestra.run.smithi042.stdout:3218 write 0x10e8733 thru 0x10e884c (0x11a bytes) 2024-09-16T13:00:03.928 INFO:teuthology.orchestra.run.smithi042.stdout:3220 read 0x2765bb1 thru 0x276b3f8 (0x5848 bytes) 2024-09-16T13:00:03.929 INFO:teuthology.orchestra.run.smithi042.stdout:3221 write 0x5495b5e thru 0x54980e7 (0x258a bytes) 2024-09-16T13:00:03.950 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:03.950 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:03.957 INFO:teuthology.orchestra.run.smithi042.stdout:3222 trunc from 0xc8c5bf5 to 0x1102bd4 2024-09-16T13:00:03.960 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:00:03.992 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:03.986+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:04.015 INFO:teuthology.orchestra.run.smithi042.stdout:3223 write 0xe83344e thru 0xe83b495 (0x8048 bytes) 2024-09-16T13:00:04.021 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.021 INFO:teuthology.orchestra.run.smithi042.stdout:3226 write 0x6f08a9e thru 0x6f120d5 (0x9638 bytes) 2024-09-16T13:00:04.025 INFO:teuthology.orchestra.run.smithi042.stdout:3227 read 0xe477d64 thru 0xe4854f3 (0xd790 bytes) 2024-09-16T13:00:04.026 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:04.027 INFO:teuthology.orchestra.run.smithi042.stdout:3228 read 0x69fb23d thru 0x69ff399 (0x415d bytes) 2024-09-16T13:00:04.027 INFO:teuthology.orchestra.run.smithi042.stdout:3229 punch from 0x5c30777 to 0x5c3bd7c, (0xb605 bytes) 2024-09-16T13:00:04.027 INFO:teuthology.orchestra.run.smithi042.stdout:3231 writesame 0x52d1eb8 thru 0x52d32af data_size 0x24(0x13f8 bytes) 2024-09-16T13:00:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:3232 write 0x7969ce8 thru 0x796ae16 (0x112f bytes) 2024-09-16T13:00:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:3234 punch from 0x2ccb3f5 to 0x2cda4df, (0xf0ea bytes) 2024-09-16T13:00:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.051 INFO:teuthology.orchestra.run.smithi042.stdout:3237 trunc from 0xe83b496 to 0x4eba91e 2024-09-16T13:00:04.057 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:04.057 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:04.063 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:04.066 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:00:04.067 INFO:teuthology.orchestra.run.smithi042.stdout:3238 read 0x1250094 thru 0x12567f7 (0x6764 bytes) 2024-09-16T13:00:04.067 INFO:teuthology.orchestra.run.smithi042.stdout:3241 read 0x2509ffd thru 0x251323a (0x923e bytes) 2024-09-16T13:00:04.067 INFO:teuthology.orchestra.run.smithi042.stdout:3242 write 0x1ef7587 thru 0x1efdcfa (0x6774 bytes) 2024-09-16T13:00:04.074 INFO:teuthology.orchestra.run.smithi042.stdout:3243 trunc from 0x4eba91e to 0x35cbef2 2024-09-16T13:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout:3244 punch from 0x265db6b to 0x266d610, (0xfaa5 bytes) 2024-09-16T13:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:04.110 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:04.106+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout:3245 write 0xe221e37 thru 0xe22c553 (0xa71d bytes) 2024-09-16T13:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout:3246 punch from 0x3239b88 to 0x324439c, (0xa814 bytes) 2024-09-16T13:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout:3248 write 0x5d811a2 thru 0x5d8d798 (0xc5f7 bytes) 2024-09-16T13:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout:3251 trunc from 0xe22c554 to 0x69051c7 2024-09-16T13:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout:3252 read 0x1b98116 thru 0x1ba2903 (0xa7ee bytes) 2024-09-16T13:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout:3257 read 0x29e89f8 thru 0x29e8e59 (0x462 bytes) 2024-09-16T13:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout:3258 trunc from 0x69051c7 to 0x38f34f3 2024-09-16T13:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:04.168 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:590 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T13:00:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.825406270000002, 2024-09-16T13:00:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034258392999999998, 2024-09-16T13:00:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 590 2024-09-16T13:00:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339271+0000", 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339281+0000", 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6670000000000008e-06 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339307+0000", 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5907000000000001e-05 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339317+0000", 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0356999999999999e-05 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339412+0000", 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4248999999999998e-05 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339533+0000", 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012184900000000001 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.347489+0000", 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079559599999999998 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373423+0000", 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025933971 2024-09-16T13:00:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373435+0000", 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1924e-05 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373492+0000", 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6842999999999998e-05 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373528+0000", 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5063 4.13 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22711298~51120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.855261472000002, 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111447293, 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5063 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309416+0000", 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358861+0000", 2024-09-16T13:00:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049444717999999999 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358893+0000", 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2279e-05 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358902+0000", 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4390000000000006e-06 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359062+0000", 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016014100000000001 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359330+0000", 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026734699999999997 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.362566+0000", 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032358769999999999 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.417541+0000", 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054975345000000002 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420706+0000", 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031651380000000001 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420790+0000", 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4283999999999995e-05 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420862+0000", 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1138999999999998e-05 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5225 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.360106152, 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027003329999999999, 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5225 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804571+0000", 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6309999999999999e-06 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804579+0000", 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804600+0000", 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1114000000000001e-05 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804613+0000", 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3040000000000001e-05 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804713+0000", 2024-09-16T13:00:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9854999999999999e-05 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804844+0000", 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130963 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.810955+0000", 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061105880000000001 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831495+0000", 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020540467 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831507+0000", 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831544+0000", 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7413000000000002e-05 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831573+0000", 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8836e-05 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5504 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.696402546000002, 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020881448, 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5504 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468276+0000", 2024-09-16T13:00:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8370000000000001e-06 2024-09-16T13:00:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468290+0000", 2024-09-16T13:00:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.383e-05 2024-09-16T13:00:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468327+0000", 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468341+0000", 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.331e-05 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468439+0000", 2024-09-16T13:00:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8746999999999995e-05 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468593+0000", 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015343999999999999 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.472806+0000", 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042129849999999998 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488823+0000", 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016017686999999999 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488936+0000", 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112074 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489029+0000", 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3801000000000005e-05 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489155+0000", 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125248 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8354 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.883308476, 2024-09-16T13:00:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016884742000000001, 2024-09-16T13:00:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8354 2024-09-16T13:00:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281370+0000", 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3159999999999999e-06 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281382+0000", 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281413+0000", 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1519000000000001e-05 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281435+0000", 2024-09-16T13:00:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1812999999999999e-05 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292521+0000", 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011085915999999999 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292628+0000", 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107311 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293108+0000", 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047976600000000001 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297972+0000", 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048640200000000001 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298174+0000", 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020222 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298226+0000", 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2058000000000002e-05 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298252+0000", 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5743000000000001e-05 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8677 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.154717506000001, 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01642155, 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8677 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009959+0000", 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2899999999999999e-07 2024-09-16T13:00:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009961+0000", 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3760000000000002e-06 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009965+0000", 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009968+0000", 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1190000000000001e-06 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023570+0000", 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013601317999999999 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023695+0000", 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012563400000000001 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024191+0000", 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000495434 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026285+0000", 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020948360000000001 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026295+0000", 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8889999999999997e-06 2024-09-16T13:00:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026338+0000", 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2253999999999999e-05 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026380+0000", 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2153000000000001e-05 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8678 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.154703254000001, 2024-09-16T13:00:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020138508999999999, 2024-09-16T13:00:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8678 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009973+0000", 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3000000000000002e-07 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009975+0000", 2024-09-16T13:00:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4739999999999999e-06 2024-09-16T13:00:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009980+0000", 2024-09-16T13:00:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4399999999999998e-06 2024-09-16T13:00:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009983+0000", 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1209999999999998e-06 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025464+0000", 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015480918 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025536+0000", 2024-09-16T13:00:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2212000000000006e-05 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025768+0000", 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000232186 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029964+0000", 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041957899999999996 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029981+0000", 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6634e-05 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030070+0000", 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9487999999999994e-05 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030111+0000", 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0915999999999999e-05 2024-09-16T13:00:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8718 4.1b 4:dbb4ad27:::rbd_data.11b55eb65e78.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.078388293, 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028260378999999999, 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8718 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086290+0000", 2024-09-16T13:00:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1950000000000002e-06 2024-09-16T13:00:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086300+0000", 2024-09-16T13:00:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0458999999999999e-05 2024-09-16T13:00:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086331+0000", 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1117e-05 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086344+0000", 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3191e-05 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086423+0000", 2024-09-16T13:00:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8288999999999995e-05 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086530+0000", 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010720799999999999 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.091612+0000", 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050817429999999997 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114274+0000", 2024-09-16T13:00:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022662832000000001 2024-09-16T13:00:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114386+0000", 2024-09-16T13:00:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011155899999999999 2024-09-16T13:00:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114475+0000", 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9043999999999997e-05 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114548+0000", 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2742000000000003e-05 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24293 4.1b 4:d9b96624:::rbd_data.11b59afb69bb.0000000000000000:16 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470599+0000", 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6940771379999999, 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015978698, 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24293 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470599+0000", 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470599+0000", 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470599+0000", 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6e-07 2024-09-16T13:00:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470603+0000", 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9349999999999996e-06 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470606+0000", 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1109999999999999e-06 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470609+0000", 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9179999999999998e-06 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486410+0000", 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015801116 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486485+0000", 2024-09-16T13:00:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5469000000000005e-05 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486577+0000", 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889000000000001e-05 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24294 4.1b 4:d9b96624:::rbd_data.11b59afb69bb.0000000000000000:16 [read 1413120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6940647970000002, 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016422685999999999, 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24294 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5499999999999999e-07 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470612+0000", 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1790000000000001e-06 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470615+0000", 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0580000000000002e-06 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470618+0000", 2024-09-16T13:00:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.103e-06 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486878+0000", 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016260398999999998 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486954+0000", 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5763e-05 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487034+0000", 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9929e-05 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24327 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1310720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470764+0000", 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6939115850000004, 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015921840999999999, 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24327 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470764+0000", 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470764+0000", 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470764+0000", 2024-09-16T13:00:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:00:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470771+0000", 2024-09-16T13:00:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2729999999999999e-06 2024-09-16T13:00:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470777+0000", 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7599999999999999e-06 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470782+0000", 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.767e-06 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486598+0000", 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015815949999999999 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486641+0000", 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2586000000000002e-05 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486686+0000", 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5141e-05 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24328 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1339392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6938889399999999, 2024-09-16T13:00:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016079547999999999, 2024-09-16T13:00:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24328 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0699999999999998e-07 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470789+0000", 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9659999999999999e-06 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470795+0000", 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4110000000000004e-06 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470798+0000", 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5499999999999999e-06 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486710+0000", 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015911536 2024-09-16T13:00:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486790+0000", 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9912000000000003e-05 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486866+0000", 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6766000000000003e-05 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24329 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1368064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6938637019999998, 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016167826999999999, 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24329 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9999999999999998e-07 2024-09-16T13:00:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470815+0000", 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6900000000000001e-06 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470821+0000", 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9050000000000002e-06 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470823+0000", 2024-09-16T13:00:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486891+0000", 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016068000999999998 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486933+0000", 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1711000000000001e-05 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486980+0000", 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6643e-05 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24330 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6938500919999999, 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016312199999999999, 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24330 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6199999999999999e-07 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470827+0000", 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.333e-06 2024-09-16T13:00:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470831+0000", 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.348e-06 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470832+0000", 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.849e-06 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486993+0000", 2024-09-16T13:00:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016160834999999998 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487060+0000", 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6748000000000005e-05 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487138+0000", 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7824999999999994e-05 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24331 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1425408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6938413949999997, 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016546297000000001, 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24331 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470835+0000", 2024-09-16T13:00:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4999999999999999e-07 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470836+0000", 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2869999999999999e-06 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470839+0000", 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0479999999999999e-06 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470841+0000", 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0090000000000002e-06 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487212+0000", 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016370609000000001 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487288+0000", 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6584999999999999e-05 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487381+0000", 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2509000000000005e-05 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24358 4.1b 4:d9b96624:::rbd_data.11b59afb69bb.0000000000000000:16 [read 1417216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6926606819999996, 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015994596, 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24358 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472016+0000", 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5199999999999996e-07 2024-09-16T13:00:04.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472017+0000", 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4249999999999999e-06 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472021+0000", 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.596e-06 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472024+0000", 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5860000000000001e-06 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487784+0000", 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015760066999999999 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487912+0000", 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012761900000000001 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488010+0000", 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7651000000000004e-05 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24393 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1372160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6923609480000001, 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016028144000000001, 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.258 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24393 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9900000000000001e-07 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472317+0000", 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.547e-06 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472321+0000", 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8970000000000001e-06 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472325+0000", 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7759999999999997e-06 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488206+0000", 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015881025 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488272+0000", 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5982000000000006e-05 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488343+0000", 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0518000000000005e-05 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24394 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6923475989999996, 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016119201, 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.263 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24394 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472329+0000", 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9500000000000003e-07 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472330+0000", 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6360000000000001e-06 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472334+0000", 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7730000000000001e-06 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472337+0000", 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7420000000000002e-06 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488362+0000", 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016025094 2024-09-16T13:00:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488405+0000", 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3438e-05 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488447+0000", 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2023000000000001e-05 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24395 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1429504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6923364579999998, 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016957005000000001, 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24395 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472340+0000", 2024-09-16T13:00:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2099999999999997e-07 2024-09-16T13:00:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472341+0000", 2024-09-16T13:00:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4190000000000001e-06 2024-09-16T13:00:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472345+0000", 2024-09-16T13:00:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8139999999999998e-06 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472348+0000", 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5780000000000001e-06 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489143+0000", 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016795244000000001 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489213+0000", 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0438000000000006e-05 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489296+0000", 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2991000000000006e-05 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:42621 4.e 4:744fcf8a:::rbd_data.11b52dde2e.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:01.415676+0000", 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.7489998719999997, 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034249963000000001, 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 42621 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415676+0000", 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415676+0000", 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415677+0000", 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.564e-06 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415685+0000", 2024-09-16T13:00:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9060000000000002e-06 2024-09-16T13:00:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415710+0000", 2024-09-16T13:00:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.421e-05 2024-09-16T13:00:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415718+0000", 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7930000000000003e-06 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415779+0000", 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0191e-05 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415887+0000", 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108936 2024-09-16T13:00:04.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.423612+0000", 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077250340000000004 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449730+0000", 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026117745000000001 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449792+0000", 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2277e-05 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449860+0000", 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7794999999999997e-05 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449926+0000", 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5512000000000006e-05 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.278 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.279 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.279 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:04.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T13:00:04.280 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:04.281 INFO:teuthology.orchestra.run.smithi042.stdout:3259 writesame 0xc28285e thru 0xc28687b data_size 0x2(0x401e bytes) 2024-09-16T13:00:04.281 INFO:teuthology.orchestra.run.smithi042.stdout:3262 trunc from 0xc28687c to 0x58444d7 2024-09-16T13:00:04.281 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:04.281 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:04.281 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:04.282 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:04.282 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:04.282 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:00:04.282 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:04.282 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:04.282 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:04.282 INFO:teuthology.orchestra.run.smithi042.stdout:3263 writesame 0x144a41e thru 0x144d84b data_size 0x1(0x342e bytes) 2024-09-16T13:00:04.283 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:04.284 INFO:teuthology.orchestra.run.smithi042.stdout:3264 punch from 0x2e866d9 to 0x2e8b08a, (0x49b1 bytes) 2024-09-16T13:00:04.284 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.284 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.284 INFO:teuthology.orchestra.run.smithi042.stdout:3267 write 0xd2b3d44 thru 0xd2befd3 (0xb290 bytes) 2024-09-16T13:00:04.284 INFO:teuthology.orchestra.run.smithi042.stdout:3268 punch from 0x6130a24 to 0x6131c98, (0x1274 bytes) 2024-09-16T13:00:04.284 INFO:teuthology.orchestra.run.smithi042.stdout:3270 trunc from 0xd2befd4 to 0x5b419b0 2024-09-16T13:00:04.285 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:04.271+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:04.288 INFO:teuthology.orchestra.run.smithi042.stdout:3271 read 0x201d42 thru 0x20f4b5 (0xd774 bytes) 2024-09-16T13:00:04.289 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.304 INFO:teuthology.orchestra.run.smithi042.stdout:3273 writesame 0x7a0f3eb thru 0x7a13300 data_size 0x22(0x3f16 bytes) 2024-09-16T13:00:04.326 INFO:teuthology.orchestra.run.smithi042.stdout:3274 write 0x97ab5f9 thru 0x97b9e07 (0xe80f bytes) 2024-09-16T13:00:04.329 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.329 INFO:teuthology.orchestra.run.smithi042.stdout:3277 punch from 0x5324419 to 0x532ca8a, (0x8671 bytes) 2024-09-16T13:00:04.329 INFO:teuthology.orchestra.run.smithi042.stdout:3280 write 0x5593f3d thru 0x5598641 (0x4705 bytes) 2024-09-16T13:00:04.347 INFO:teuthology.orchestra.run.smithi042.stdout:3281 writesame 0xb1e5e4c thru 0xb1ec3d8 data_size 0x1(0x658d bytes) 2024-09-16T13:00:04.350 INFO:teuthology.orchestra.run.smithi042.stdout:3283 read 0x858b421 thru 0x8597eec (0xcacc bytes) 2024-09-16T13:00:04.351 INFO:teuthology.orchestra.run.smithi042.stdout:3284 read 0xd54883 thru 0xd603fd (0xbb7b bytes) 2024-09-16T13:00:04.352 INFO:teuthology.orchestra.run.smithi042.stdout:3285 read 0x8c48de2 thru 0x8c51412 (0x8631 bytes) 2024-09-16T13:00:04.353 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.353 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.353 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.353 INFO:teuthology.orchestra.run.smithi042.stdout:3291 write 0xaf529c8 thru 0xaf5a220 (0x7859 bytes) 2024-09-16T13:00:04.356 INFO:teuthology.orchestra.run.smithi042.stdout:3292 read 0x796c200 thru 0x79718bd (0x56be bytes) 2024-09-16T13:00:04.357 INFO:teuthology.orchestra.run.smithi042.stdout:3294 write 0xa5b54e4 thru 0xa5ba87d (0x539a bytes) 2024-09-16T13:00:04.361 INFO:teuthology.orchestra.run.smithi042.stdout:3296 write 0x80b122 thru 0x817698 (0xc577 bytes) 2024-09-16T13:00:04.365 INFO:teuthology.orchestra.run.smithi042.stdout:3297 writesame 0xafbb7ae thru 0xafbfc01 data_size 0x2(0x4454 bytes) 2024-09-16T13:00:04.373 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.373 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.381 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:04.375+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:04.388 INFO:teuthology.orchestra.run.smithi042.stdout:3302 write 0xc39cd61 thru 0xc3a039f (0x363f bytes) 2024-09-16T13:00:04.391 INFO:teuthology.orchestra.run.smithi042.stdout:3303 punch from 0x54c1299 to 0x54d0b15, (0xf87c bytes) 2024-09-16T13:00:04.391 INFO:teuthology.orchestra.run.smithi042.stdout:3304 read 0x8f15073 thru 0x8f1b2cc (0x625a bytes) 2024-09-16T13:00:04.399 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:04.399 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:04.399 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:04.399 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:04.403 INFO:teuthology.orchestra.run.smithi042.stdout:3305 trunc from 0xc3a03a0 to 0x50c045a 2024-09-16T13:00:04.410 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:04.422 INFO:teuthology.orchestra.run.smithi042.stdout:3306 trunc from 0x50c045a to 0x1dac6e0 2024-09-16T13:00:04.440 INFO:teuthology.orchestra.run.smithi042.stdout:3310 read 0x146985d thru 0x146b943 (0x20e7 bytes) 2024-09-16T13:00:04.442 INFO:teuthology.orchestra.run.smithi042.stdout:3311 trunc from 0x1dac6e0 to 0x13cd6d0 2024-09-16T13:00:04.482 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:04.475+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:04.489 INFO:teuthology.orchestra.run.smithi042.stdout:3312 writesame 0xd07c9c1 thru 0xd080ac2 data_size 0x35(0x4102 bytes) 2024-09-16T13:00:04.489 INFO:teuthology.orchestra.run.smithi042.stdout:3313 punch from 0x3f4c68a to 0x3f5b956, (0xf2cc bytes) 2024-09-16T13:00:04.489 INFO:teuthology.orchestra.run.smithi042.stdout:3314 write 0xde0e52 thru 0xdef54b (0xe6fa bytes) 2024-09-16T13:00:04.492 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.492 INFO:teuthology.orchestra.run.smithi042.stdout:3316 read 0x1bbb68d thru 0x1bc875d (0xd0d1 bytes) 2024-09-16T13:00:04.506 INFO:teuthology.orchestra.run.smithi042.stdout:3320 write 0xe485f22 thru 0xe4924a5 (0xc584 bytes) 2024-09-16T13:00:04.509 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:04.509 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:04.510 INFO:teuthology.orchestra.run.smithi042.stdout:3321 punch from 0x7a77b6e to 0x7a860e6, (0xe578 bytes) 2024-09-16T13:00:04.510 INFO:teuthology.orchestra.run.smithi042.stdout:3322 write 0x2fcb597 thru 0x2fcdadf (0x2549 bytes) 2024-09-16T13:00:04.515 INFO:teuthology.orchestra.run.smithi042.stdout:3323 writesame 0xe123aa0 thru 0xe12de87 data_size 0x14(0xa3e8 bytes) 2024-09-16T13:00:04.518 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:04.536 INFO:teuthology.orchestra.run.smithi042.stdout:3324 trunc from 0xe4924a6 to 0x2a1c30c 2024-09-16T13:00:04.580 INFO:teuthology.orchestra.run.smithi042.stdout:3326 write 0xd3215c2 thru 0xd32521c (0x3c5b bytes) 2024-09-16T13:00:04.584 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:04.578+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:04.586 INFO:teuthology.orchestra.run.smithi042.stdout:3327 writesame 0x886d88e thru 0x8874454 data_size 0x11(0x6bc7 bytes) 2024-09-16T13:00:04.591 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.591 INFO:teuthology.orchestra.run.smithi042.stdout:3330 writesame 0x6a21c13 thru 0x6a276e8 data_size 0x16(0x5ad6 bytes) 2024-09-16T13:00:04.596 INFO:teuthology.orchestra.run.smithi042.stdout:3331 trunc from 0xd32521d to 0xd553af3 2024-09-16T13:00:04.605 INFO:teuthology.orchestra.run.smithi042.stdout:3333 write 0x147b8af thru 0x147c58e (0xce0 bytes) 2024-09-16T13:00:04.610 INFO:teuthology.orchestra.run.smithi042.stdout:3334 read 0x95e6a82 thru 0x95e8348 (0x18c7 bytes) 2024-09-16T13:00:04.610 INFO:teuthology.orchestra.run.smithi042.stdout:3335 write 0x36c44ee thru 0x36c80f1 (0x3c04 bytes) 2024-09-16T13:00:04.615 INFO:teuthology.orchestra.run.smithi042.stdout:3336 punch from 0x47d771b to 0x47dfbb5, (0x849a bytes) 2024-09-16T13:00:04.615 INFO:teuthology.orchestra.run.smithi042.stdout:3338 write 0xe5f10 thru 0xf2e45 (0xcf36 bytes) 2024-09-16T13:00:04.617 INFO:teuthology.orchestra.run.smithi042.stdout:3339 writesame 0xcec97c1 thru 0xced1c50 data_size 0x38(0x8490 bytes) 2024-09-16T13:00:04.621 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8419 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.214259687999999, 2024-09-16T13:00:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040293592000000003, 2024-09-16T13:00:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8419 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402894+0000", 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402907+0000", 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402940+0000", 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3436000000000003e-05 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402953+0000", 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2840000000000001e-05 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403104+0000", 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150712 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403202+0000", 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8403e-05 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.411015+0000", 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078133079999999997 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443018+0000", 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032002348999999999 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443075+0000", 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7703e-05 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443141+0000", 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5705999999999997e-05 2024-09-16T13:00:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443185+0000", 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4152e-05 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:8419 4.7 e54/40 ondisk, result = 0)", 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.431024+0000", 2024-09-16T13:00:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.186127135, 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012172095000000001, 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431024+0000", 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431024+0000", 2024-09-16T13:00:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431025+0000", 2024-09-16T13:00:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3820000000000001e-06 2024-09-16T13:00:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431032+0000", 2024-09-16T13:00:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4389999999999997e-06 2024-09-16T13:00:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431054+0000", 2024-09-16T13:00:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2008e-05 2024-09-16T13:00:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.431064+0000", 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-05 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443061+0000", 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011997017 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443069+0000", 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7309999999999993e-06 2024-09-16T13:00:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443196+0000", 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012701800000000001 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8612 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.847921+0000", 2024-09-16T13:00:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.769230023999999, 2024-09-16T13:00:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039491146999999997, 2024-09-16T13:00:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8612 2024-09-16T13:00:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847921+0000", 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847921+0000", 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847924+0000", 2024-09-16T13:00:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0199999999999999e-06 2024-09-16T13:00:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847936+0000", 2024-09-16T13:00:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1579999999999999e-05 2024-09-16T13:00:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847968+0000", 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2033000000000003e-05 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.847986+0000", 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7839e-05 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.848125+0000", 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013962800000000001 2024-09-16T13:00:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.848277+0000", 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015126199999999999 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.856740+0000", 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084638479999999995 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887229+0000", 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030488491 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887253+0000", 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4409e-05 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887347+0000", 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3467999999999993e-05 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.887412+0000", 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5568999999999995e-05 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:15878 4.15 4:af50264e:::rbd_data.11b5c2c64266.0000000000000129:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.552855+0000", 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.064295857999999, 2024-09-16T13:00:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011857385, 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 15878 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552855+0000", 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552855+0000", 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552858+0000", 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2960000000000001e-06 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552869+0000", 2024-09-16T13:00:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1197e-05 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552902+0000", 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3219000000000002e-05 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.552915+0000", 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.257e-05 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.553032+0000", 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117784 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.553314+0000", 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028161000000000001 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.555090+0000", 2024-09-16T13:00:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001776047 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.563585+0000", 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084951169999999999 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.564623+0000", 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001037638 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.564683+0000", 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9889999999999997e-05 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.564713+0000", 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0017000000000001e-05 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16273 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754081+0000", 2024-09-16T13:00:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.863070524999999, 2024-09-16T13:00:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010285459, 2024-09-16T13:00:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16273 2024-09-16T13:00:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754081+0000", 2024-09-16T13:00:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754081+0000", 2024-09-16T13:00:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754081+0000", 2024-09-16T13:00:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3000000000000001e-07 2024-09-16T13:00:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754084+0000", 2024-09-16T13:00:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.525e-06 2024-09-16T13:00:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754091+0000", 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7319999999999998e-06 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754096+0000", 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7099999999999998e-06 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764261+0000", 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010164566999999999 2024-09-16T13:00:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764304+0000", 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3316000000000003e-05 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764366+0000", 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2179000000000002e-05 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16274 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 294912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754101+0000", 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.86305033, 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010382317, 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16274 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754101+0000", 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754101+0000", 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754102+0000", 2024-09-16T13:00:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2099999999999996e-07 2024-09-16T13:00:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754105+0000", 2024-09-16T13:00:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9950000000000001e-06 2024-09-16T13:00:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754113+0000", 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4579999999999993e-06 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754118+0000", 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.843e-06 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764384+0000", 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010266566 2024-09-16T13:00:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764428+0000", 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3377999999999997e-05 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764483+0000", 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5455999999999999e-05 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16275 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754123+0000", 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.863028448, 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010473434, 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16275 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754123+0000", 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754123+0000", 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754123+0000", 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7300000000000001e-07 2024-09-16T13:00:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754139+0000", 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5458999999999999e-05 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754147+0000", 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6109999999999994e-06 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754153+0000", 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6400000000000002e-06 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764501+0000", 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010347656 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764543+0000", 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2088000000000001e-05 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764596+0000", 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3507000000000001e-05 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16276 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754158+0000", 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862992831, 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010575052999999999, 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16276 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754158+0000", 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754158+0000", 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754159+0000", 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.44e-07 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754163+0000", 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9360000000000001e-06 2024-09-16T13:00:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754171+0000", 2024-09-16T13:00:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7470000000000005e-06 2024-09-16T13:00:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754177+0000", 2024-09-16T13:00:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4749999999999998e-06 2024-09-16T13:00:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764614+0000", 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010436543 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764655+0000", 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1628999999999997e-05 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764733+0000", 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8178999999999998e-05 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16277 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754182+0000", 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862968951999999, 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010745922, 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16277 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754182+0000", 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754182+0000", 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754183+0000", 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0900000000000002e-07 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754186+0000", 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3440000000000001e-06 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754194+0000", 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1729999999999995e-06 2024-09-16T13:00:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754199+0000", 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0010000000000001e-06 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764784+0000", 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010584748 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764846+0000", 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2292000000000006e-05 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764928+0000", 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1854999999999995e-05 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16295 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 794624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754204+0000", 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862946979, 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010835667, 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16295 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754204+0000", 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754204+0000", 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754205+0000", 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9999999999999996e-07 2024-09-16T13:00:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754208+0000", 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.492e-06 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754216+0000", 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9859999999999994e-06 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754221+0000", 2024-09-16T13:00:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3530000000000002e-06 2024-09-16T13:00:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764947+0000", 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010725656 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.764994+0000", 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256000000000003e-05 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765040+0000", 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6023999999999998e-05 2024-09-16T13:00:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16296 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754246+0000", 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862905506000001, 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011016732, 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16296 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754246+0000", 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754246+0000", 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754246+0000", 2024-09-16T13:00:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6300000000000005e-07 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754249+0000", 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2900000000000001e-06 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754254+0000", 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8270000000000003e-06 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754258+0000", 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765106+0000", 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010848343 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765184+0000", 2024-09-16T13:00:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8220000000000007e-05 2024-09-16T13:00:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765262+0000", 2024-09-16T13:00:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8039000000000003e-05 2024-09-16T13:00:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16297 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 942080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754263+0000", 2024-09-16T13:00:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862888671, 2024-09-16T13:00:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011169742, 2024-09-16T13:00:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16297 2024-09-16T13:00:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754263+0000", 2024-09-16T13:00:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754263+0000", 2024-09-16T13:00:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754263+0000", 2024-09-16T13:00:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754265+0000", 2024-09-16T13:00:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.798e-06 2024-09-16T13:00:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754269+0000", 2024-09-16T13:00:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0659999999999997e-06 2024-09-16T13:00:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754272+0000", 2024-09-16T13:00:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1370000000000002e-06 2024-09-16T13:00:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765310+0000", 2024-09-16T13:00:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011038031 2024-09-16T13:00:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765372+0000", 2024-09-16T13:00:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1989999999999994e-05 2024-09-16T13:00:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765432+0000", 2024-09-16T13:00:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0371999999999997e-05 2024-09-16T13:00:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16298 4.8 4:1045a441:::rbd_data.11b528bff1d7.0000000000000005:13 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754275+0000", 2024-09-16T13:00:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862876565000001, 2024-09-16T13:00:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011358536000000001, 2024-09-16T13:00:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16298 2024-09-16T13:00:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754275+0000", 2024-09-16T13:00:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754275+0000", 2024-09-16T13:00:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754275+0000", 2024-09-16T13:00:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9999999999999998e-07 2024-09-16T13:00:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754277+0000", 2024-09-16T13:00:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4780000000000001e-06 2024-09-16T13:00:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754281+0000", 2024-09-16T13:00:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0820000000000001e-06 2024-09-16T13:00:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754284+0000", 2024-09-16T13:00:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9050000000000001e-06 2024-09-16T13:00:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765506+0000", 2024-09-16T13:00:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01122248 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765572+0000", 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5458999999999998e-05 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765633+0000", 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1631999999999995e-05 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16299 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754296+0000", 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862855334000001, 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011521811, 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16299 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754296+0000", 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754296+0000", 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754296+0000", 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.96e-07 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754298+0000", 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754303+0000", 2024-09-16T13:00:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1420000000000001e-06 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754307+0000", 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1470000000000001e-06 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765675+0000", 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011368417 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765737+0000", 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1586000000000003e-05 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765818+0000", 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1193999999999995e-05 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16300 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754309+0000", 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862841787000001, 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011692226999999999, 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16300 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754309+0000", 2024-09-16T13:00:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754309+0000", 2024-09-16T13:00:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754310+0000", 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2800000000000003e-07 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754311+0000", 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754316+0000", 2024-09-16T13:00:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1540000000000004e-06 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754318+0000", 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7939999999999998e-06 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765846+0000", 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011527720999999999 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.765909+0000", 2024-09-16T13:00:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3056000000000001e-05 2024-09-16T13:00:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766002+0000", 2024-09-16T13:00:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2588000000000002e-05 2024-09-16T13:00:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16301 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754321+0000", 2024-09-16T13:00:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862830369999999, 2024-09-16T13:00:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011860290000000001, 2024-09-16T13:00:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16301 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754321+0000", 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754321+0000", 2024-09-16T13:00:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754321+0000", 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2099999999999998e-07 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754323+0000", 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4139999999999999e-06 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754327+0000", 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9820000000000002e-06 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754329+0000", 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.582e-06 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766038+0000", 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011708435999999999 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766099+0000", 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1699999999999995e-05 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766181+0000", 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1854999999999995e-05 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16302 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754332+0000", 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862818883999999, 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012047413999999999, 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16302 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754332+0000", 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754332+0000", 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754333+0000", 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.39e-07 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754334+0000", 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3570000000000001e-06 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754338+0000", 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.233e-06 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754341+0000", 2024-09-16T13:00:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7240000000000001e-06 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766224+0000", 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011882547 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766291+0000", 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7347000000000004e-05 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766380+0000", 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8867000000000001e-05 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16303 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754344+0000", 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862807598, 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012220444, 2024-09-16T13:00:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16303 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754344+0000", 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754344+0000", 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754344+0000", 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754345+0000", 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0899999999999999e-06 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754349+0000", 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9790000000000004e-06 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754356+0000", 2024-09-16T13:00:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8279999999999997e-06 2024-09-16T13:00:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766418+0000", 2024-09-16T13:00:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012062226000000001 2024-09-16T13:00:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766488+0000", 2024-09-16T13:00:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9438999999999997e-05 2024-09-16T13:00:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766564+0000", 2024-09-16T13:00:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6514999999999995e-05 2024-09-16T13:00:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:16304 4.8 4:15c31f4a:::rbd_data.11b528bff1d7.0000000000000009:13 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:50.754361+0000", 2024-09-16T13:00:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.862789801, 2024-09-16T13:00:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012417071999999999, 2024-09-16T13:00:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16304 2024-09-16T13:00:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754361+0000", 2024-09-16T13:00:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754361+0000", 2024-09-16T13:00:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754362+0000", 2024-09-16T13:00:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754363+0000", 2024-09-16T13:00:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.28e-06 2024-09-16T13:00:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754368+0000", 2024-09-16T13:00:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.605e-06 2024-09-16T13:00:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.754371+0000", 2024-09-16T13:00:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9239999999999999e-06 2024-09-16T13:00:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766605+0000", 2024-09-16T13:00:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012234282000000001 2024-09-16T13:00:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766701+0000", 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6175999999999995e-05 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:50.766779+0000", 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7454999999999996e-05 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:42521 4.8 4:15657af1:::rbd_data.11b52dde2e.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:01.158529+0000", 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.4586223789999999, 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035255728, 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 42521 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158529+0000", 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158529+0000", 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158530+0000", 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.108e-06 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158536+0000", 2024-09-16T13:00:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1290000000000001e-06 2024-09-16T13:00:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158554+0000", 2024-09-16T13:00:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7833e-05 2024-09-16T13:00:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158561+0000", 2024-09-16T13:00:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5159999999999997e-06 2024-09-16T13:00:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158700+0000", 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138071 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.158833+0000", 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013376499999999999 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.169342+0000", 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010508999999999999 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.193701+0000", 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024358956000000001 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.193712+0000", 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1093e-05 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.193754+0000", 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1995e-05 2024-09-16T13:00:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.193785+0000", 2024-09-16T13:00:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0261999999999999e-05 2024-09-16T13:00:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:04.741 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:04.741 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:04.743 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:04.744 INFO:teuthology.orchestra.run.smithi042.stdout:3340 read 0xc294a2a thru 0xc294fda (0x5b1 bytes) 2024-09-16T13:00:04.744 INFO:teuthology.orchestra.run.smithi042.stdout:3341 write 0x613a8ea thru 0x614223f (0x7956 bytes) 2024-09-16T13:00:04.744 INFO:teuthology.orchestra.run.smithi042.stdout:3342 write 0x17a9f75 thru 0x17abc84 (0x1d10 bytes) 2024-09-16T13:00:04.744 INFO:teuthology.orchestra.run.smithi042.stdout:3343 read 0x898bf4e thru 0x8997ebe (0xbf71 bytes) 2024-09-16T13:00:04.744 INFO:teuthology.orchestra.run.smithi042.stdout:3344 read 0xfbe139 thru 0xfbe727 (0x5ef bytes) 2024-09-16T13:00:04.744 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.744 INFO:teuthology.orchestra.run.smithi042.stdout:3346 write 0x780031f thru 0x780ee6d (0xeb4f bytes) 2024-09-16T13:00:04.744 INFO:teuthology.orchestra.run.smithi042.stdout:3347 writesame 0xe421501 thru 0xe4281e4 data_size 0x2e(0x6ce4 bytes) 2024-09-16T13:00:04.745 INFO:teuthology.orchestra.run.smithi042.stdout:3348 punch from 0x97dddc5 to 0x97eb2de, (0xd519 bytes) 2024-09-16T13:00:04.745 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.745 INFO:teuthology.orchestra.run.smithi042.stdout:3350 read 0x1cd26b5 thru 0x1cdee0f (0xc75b bytes) 2024-09-16T13:00:04.745 INFO:teuthology.orchestra.run.smithi042.stdout:3351 trunc from 0xe4281e5 to 0x755d62e 2024-09-16T13:00:04.745 INFO:teuthology.orchestra.run.smithi042.stdout:3353 trunc from 0x755d62e to 0xf38690 2024-09-16T13:00:04.745 INFO:teuthology.orchestra.run.smithi042.stdout:3354 trunc from 0xf38690 to 0xaed181f 2024-09-16T13:00:04.745 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:04.735+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:04.746 INFO:teuthology.orchestra.run.smithi042.stdout:3355 write 0xd9d8a9 thru 0xda13aa (0x3b02 bytes) 2024-09-16T13:00:04.755 INFO:teuthology.orchestra.run.smithi042.stdout:3357 trunc from 0xaed181f to 0xe243859 2024-09-16T13:00:04.767 INFO:teuthology.orchestra.run.smithi042.stdout:3358 writesame 0xe0d1cc0 thru 0xe0d8d64 data_size 0x1(0x70a5 bytes) 2024-09-16T13:00:04.791 INFO:teuthology.orchestra.run.smithi042.stdout:3359 write 0xbd53e83 thru 0xbd5c454 (0x85d2 bytes) 2024-09-16T13:00:04.797 INFO:teuthology.orchestra.run.smithi042.stdout:3360 writesame 0x84b74ee thru 0x84be826 data_size 0x1(0x7339 bytes) 2024-09-16T13:00:04.822 INFO:teuthology.orchestra.run.smithi042.stdout:3365 punch from 0x4a0dc3a to 0x4a189fd, (0xadc3 bytes) 2024-09-16T13:00:04.822 INFO:teuthology.orchestra.run.smithi042.stdout:3366 punch from 0x12e36aa to 0x12ea8e7, (0x723d bytes) 2024-09-16T13:00:04.822 INFO:teuthology.orchestra.run.smithi042.stdout:3367 read 0x3578d97 thru 0x357a0d6 (0x1340 bytes) 2024-09-16T13:00:04.823 INFO:teuthology.orchestra.run.smithi042.stdout:3368 write 0x9780a92 thru 0x978de6a (0xd3d9 bytes) 2024-09-16T13:00:04.829 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.829 INFO:teuthology.orchestra.run.smithi042.stdout:3375 write 0x7c296ec thru 0x7c2d646 (0x3f5b bytes) 2024-09-16T13:00:04.831 INFO:teuthology.orchestra.run.smithi042.stdout:3376 writesame 0x61bda0d thru 0x61c5301 data_size 0xb(0x78f5 bytes) 2024-09-16T13:00:04.838 INFO:teuthology.orchestra.run.smithi042.stdout:3377 writesame 0x22d992d thru 0x22e1133 data_size 0x1(0x7807 bytes) 2024-09-16T13:00:04.843 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:04.838+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:04.851 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:04.852 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:04.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:04.862 INFO:teuthology.orchestra.run.smithi042.stdout:3379 write 0xd12b8be thru 0xd12fb80 (0x42c3 bytes) 2024-09-16T13:00:04.864 INFO:teuthology.orchestra.run.smithi042.stdout:3380 write 0x25d247 thru 0x267f21 (0xacdb bytes) 2024-09-16T13:00:04.868 INFO:teuthology.orchestra.run.smithi042.stdout:3381 punch from 0x6b11521 to 0x6b1bf2d, (0xaa0c bytes) 2024-09-16T13:00:04.868 INFO:teuthology.orchestra.run.smithi042.stdout:3382 write 0x5763cf7 thru 0x576a0cc (0x63d6 bytes) 2024-09-16T13:00:04.875 INFO:teuthology.orchestra.run.smithi042.stdout:3384 write 0x18d3f15 thru 0x18dd4ee (0x95da bytes) 2024-09-16T13:00:04.881 INFO:teuthology.orchestra.run.smithi042.stdout:3385 punch from 0x805112c to 0x805cca2, (0xbb76 bytes) 2024-09-16T13:00:04.881 INFO:teuthology.orchestra.run.smithi042.stdout:3387 read 0x6f0c7b4 thru 0x6f1138d (0x4bda bytes) 2024-09-16T13:00:04.881 INFO:teuthology.orchestra.run.smithi042.stdout:3389 read 0x64042bd thru 0x640ea20 (0xa764 bytes) 2024-09-16T13:00:04.910 INFO:teuthology.orchestra.run.smithi042.stdout:3390 trunc from 0xe243859 to 0xaa01b6 2024-09-16T13:00:04.938 INFO:teuthology.orchestra.run.smithi042.stdout:3391 read 0x981f23 thru 0x985eae (0x3f8c bytes) 2024-09-16T13:00:04.939 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.939 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.940 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:04.944 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:04.938+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:04.957 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:04.958 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:04.967 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:04.974 INFO:teuthology.orchestra.run.smithi042.stdout:3396 write 0xe0bb660 thru 0xe0c3ffa (0x899b bytes) 2024-09-16T13:00:04.977 INFO:teuthology.orchestra.run.smithi042.stdout:3397 read 0x89462c thru 0x895233 (0xc08 bytes) 2024-09-16T13:00:04.978 INFO:teuthology.orchestra.run.smithi042.stdout:3399 writesame 0x2001a7a thru 0x200edac data_size 0x1(0xd333 bytes) 2024-09-16T13:00:05.018 INFO:teuthology.orchestra.run.smithi042.stdout:3400 read 0x208151e thru 0x208bb0f (0xa5f2 bytes) 2024-09-16T13:00:05.039 INFO:teuthology.orchestra.run.smithi042.stdout:3401 trunc from 0xe0c3ffb to 0x5eeebdd 2024-09-16T13:00:05.045 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:05.038+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:05.071 INFO:teuthology.orchestra.run.smithi042.stdout:3402 trunc from 0x5eeebdd to 0xea1c10d 2024-09-16T13:00:05.090 INFO:teuthology.orchestra.run.smithi042.stdout:3403 trunc from 0xea1c10d to 0xa506bf9 2024-09-16T13:00:05.103 INFO:teuthology.orchestra.run.smithi042.stdout:3404 read 0x5fa1bb1 thru 0x5fa73f8 (0x5848 bytes) 2024-09-16T13:00:05.103 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.103 INFO:teuthology.orchestra.run.smithi042.stdout:3406 write 0x847a8df thru 0x8487de6 (0xd508 bytes) 2024-09-16T13:00:05.109 INFO:teuthology.orchestra.run.smithi042.stdout:3410 write 0x2de3743 thru 0x2de51d6 (0x1a94 bytes) 2024-09-16T13:00:05.114 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.746229143999997, 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T13:00:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5076 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.451911+0000", 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.658511472999997, 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020443647999999998, 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5076 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451913+0000", 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451920+0000", 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5459999999999997e-06 2024-09-16T13:00:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451938+0000", 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7581999999999999e-05 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451950+0000", 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2033e-05 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452042+0000", 2024-09-16T13:00:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2533000000000003e-05 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452251+0000", 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020887 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.456761+0000", 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045099959999999996 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472274+0000", 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015512882 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472281+0000", 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3939999999999998e-06 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472324+0000", 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2389999999999999e-05 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472355+0000", 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1436000000000002e-05 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5580 4.12 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.417388414999998, 2024-09-16T13:00:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034176870999999998, 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5580 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693037+0000", 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0650000000000001e-06 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693045+0000", 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T13:00:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693070+0000", 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5341999999999999e-05 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693081+0000", 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0611000000000001e-05 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693182+0000", 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100437 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693309+0000", 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012741400000000001 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.703090+0000", 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097814800000000004 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726908+0000", 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023817680000000001 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727023+0000", 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011449400000000001 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727130+0000", 2024-09-16T13:00:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010704099999999999 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727211+0000", 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1675000000000007e-05 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:18031 4.4 4:2649aabf:::rbd_data.11b51a3eb68b.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~5304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.9499059110000001, 2024-09-16T13:00:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94984657100000003, 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 18031 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160519+0000", 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3029999999999998e-06 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160536+0000", 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7181999999999999e-05 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160564+0000", 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971e-05 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160576+0000", 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160669+0000", 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3282999999999994e-05 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160680+0000", 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107188+0000", 2024-09-16T13:00:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94650794999999999 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107644+0000", 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00045604499999999997 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.108257+0000", 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061324700000000003 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.109811+0000", 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015538749999999999 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110175+0000", 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036419399999999998 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110285+0000", 2024-09-16T13:00:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010938900000000001 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110364+0000", 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8689000000000005e-05 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:22908 4.a 4:52996d8f:::rbd_data.11b59afb69bb.000000000000000b:16 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.411799+0000", 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6986234549999999, 2024-09-16T13:00:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011921209, 2024-09-16T13:00:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 22908 2024-09-16T13:00:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411799+0000", 2024-09-16T13:00:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411799+0000", 2024-09-16T13:00:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411800+0000", 2024-09-16T13:00:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411801+0000", 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.59e-06 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411805+0000", 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0080000000000004e-06 2024-09-16T13:00:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411808+0000", 2024-09-16T13:00:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8449999999999999e-06 2024-09-16T13:00:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.423504+0000", 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011696157 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.423626+0000", 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012167600000000001 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.423721+0000", 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4576999999999999e-05 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:22964 4.a 4:5181cc7f:::rbd_data.11b59afb69bb.000000000000000a:16 [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.411878+0000", 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.698544611, 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012066926, 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 22964 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411878+0000", 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411878+0000", 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411879+0000", 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.06e-07 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411880+0000", 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5239999999999999e-06 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411885+0000", 2024-09-16T13:00:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1339999999999997e-06 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411888+0000", 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8850000000000001e-06 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.423766+0000", 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011877655000000001 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.423877+0000", 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111439 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.423945+0000", 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7682999999999996e-05 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:22965 4.a 4:5181cc7f:::rbd_data.11b59afb69bb.000000000000000a:16 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.411891+0000", 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6985316770000001, 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012266376000000001, 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 22965 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411891+0000", 2024-09-16T13:00:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411891+0000", 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411892+0000", 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7e-07 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411893+0000", 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3659999999999999e-06 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411899+0000", 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7939999999999999e-06 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411901+0000", 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7240000000000001e-06 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.423977+0000", 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012075356000000001 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424052+0000", 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5214000000000002e-05 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424158+0000", 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105552 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:22966 4.a 4:5181cc7f:::rbd_data.11b59afb69bb.000000000000000a:16 [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.411904+0000", 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6985187479999997, 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012493957, 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 22966 2024-09-16T13:00:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411904+0000", 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411904+0000", 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411904+0000", 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3500000000000002e-07 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411906+0000", 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5260000000000001e-06 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411910+0000", 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0040000000000001e-06 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411913+0000", 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.807e-06 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424191+0000", 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012278423 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424279+0000", 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7416999999999993e-05 2024-09-16T13:00:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424398+0000", 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011944500000000001 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:22967 4.a 4:5181cc7f:::rbd_data.11b59afb69bb.000000000000000a:16 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.411915+0000", 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6985074860000005, 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012708216, 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 22967 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411915+0000", 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411915+0000", 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411916+0000", 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2699999999999999e-07 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411917+0000", 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4899999999999999e-06 2024-09-16T13:00:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411921+0000", 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9230000000000004e-06 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411924+0000", 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8030000000000001e-06 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424429+0000", 2024-09-16T13:00:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012505288999999999 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424526+0000", 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6430999999999999e-05 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424624+0000", 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7752999999999997e-05 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:22968 4.a 4:52996d8f:::rbd_data.11b59afb69bb.000000000000000b:16 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.411927+0000", 2024-09-16T13:00:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6984961250000001, 2024-09-16T13:00:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012886936999999999, 2024-09-16T13:00:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 22968 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411927+0000", 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411927+0000", 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411927+0000", 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2599999999999998e-07 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411929+0000", 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4929999999999999e-06 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411934+0000", 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9080000000000003e-06 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411937+0000", 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5889999999999999e-06 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424650+0000", 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012712973000000001 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424726+0000", 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6126999999999998e-05 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424814+0000", 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7421000000000002e-05 2024-09-16T13:00:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:22969 4.a 4:52996d8f:::rbd_data.11b59afb69bb.000000000000000b:16 [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.411940+0000", 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6984830080000002, 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013053749, 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 22969 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411940+0000", 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411940+0000", 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411940+0000", 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411942+0000", 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.404e-06 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411948+0000", 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6690000000000003e-06 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411950+0000", 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8370000000000001e-06 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424842+0000", 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012891823 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424915+0000", 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2652999999999997e-05 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.424994+0000", 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8752999999999996e-05 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:22970 4.a 4:52996d8f:::rbd_data.11b59afb69bb.000000000000000b:16 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.411953+0000", 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6984695849999998, 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013293642, 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 22970 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411953+0000", 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411953+0000", 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411954+0000", 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411955+0000", 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3480000000000001e-06 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411960+0000", 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8330000000000001e-06 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411963+0000", 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6989999999999999e-06 2024-09-16T13:00:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.425019+0000", 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013056592000000001 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.425112+0000", 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2410000000000005e-05 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.425247+0000", 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013530000000000001 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:22971 4.a 4:52996d8f:::rbd_data.11b59afb69bb.000000000000000b:16 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.411965+0000", 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6984575629999998, 2024-09-16T13:00:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013542388000000001, 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 22971 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411965+0000", 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411965+0000", 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411966+0000", 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7300000000000002e-07 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411967+0000", 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2100000000000001e-06 2024-09-16T13:00:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411971+0000", 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9530000000000001e-06 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411975+0000", 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3170000000000003e-06 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.425280+0000", 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013304373 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.425392+0000", 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112577 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.425508+0000", 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115585 2024-09-16T13:00:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:22972 4.a 4:52996d8f:::rbd_data.11b59afb69bb.000000000000000b:16 [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.411980+0000", 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.698443073, 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013748336, 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 22972 2024-09-16T13:00:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411980+0000", 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411980+0000", 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411980+0000", 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.39e-07 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411981+0000", 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.308e-06 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411985+0000", 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2024-09-16T13:00:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.411989+0000", 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.625e-06 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.425541+0000", 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013551748000000001 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.425631+0000", 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0449000000000001e-05 2024-09-16T13:00:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.425728+0000", 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6846000000000001e-05 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:30352 4.a 4:5181cc7f:::rbd_data.11b59afb69bb.000000000000000a:16 [read 3104768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.681650+0000", 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4287728890000002, 2024-09-16T13:00:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011801166, 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 30352 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681650+0000", 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681650+0000", 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681650+0000", 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681653+0000", 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.198e-06 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681658+0000", 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5010000000000004e-06 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681662+0000", 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6320000000000001e-06 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693365+0000", 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011703293 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693406+0000", 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1069000000000003e-05 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693451+0000", 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5071000000000003e-05 2024-09-16T13:00:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:30353 4.a 4:5181cc7f:::rbd_data.11b59afb69bb.000000000000000a:16 [read 3133440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.681665+0000", 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4287573939999998, 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011913837, 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 30353 2024-09-16T13:00:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681665+0000", 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681665+0000", 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681666+0000", 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7500000000000001e-07 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681668+0000", 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.795e-06 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681673+0000", 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4210000000000003e-06 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681676+0000", 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4039999999999999e-06 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693489+0000", 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011812435 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693534+0000", 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5241000000000003e-05 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693579+0000", 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5166000000000001e-05 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:30354 4.a 4:52996d8f:::rbd_data.11b59afb69bb.000000000000000b:16 [read 3031040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.681680+0000", 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4287424849999999, 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012003385, 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 30354 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681680+0000", 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681680+0000", 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681681+0000", 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1099999999999995e-07 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681683+0000", 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.888e-06 2024-09-16T13:00:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681689+0000", 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7490000000000004e-06 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681694+0000", 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7760000000000003e-06 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693601+0000", 2024-09-16T13:00:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011907091 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693645+0000", 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3037999999999997e-05 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693684+0000", 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9232e-05 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:30355 4.a 4:52996d8f:::rbd_data.11b59afb69bb.000000000000000b:16 [read 3059712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.681698+0000", 2024-09-16T13:00:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4287245239999997, 2024-09-16T13:00:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012081303, 2024-09-16T13:00:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 30355 2024-09-16T13:00:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681698+0000", 2024-09-16T13:00:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681698+0000", 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681699+0000", 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.65e-07 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681701+0000", 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9240000000000001e-06 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681706+0000", 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3630000000000002e-06 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681710+0000", 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.534e-06 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693700+0000", 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011990181000000001 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693743+0000", 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3223999999999998e-05 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693780+0000", 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6711999999999997e-05 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:30356 4.a 4:52996d8f:::rbd_data.11b59afb69bb.000000000000000b:16 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.681715+0000", 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.428708307, 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012158733999999999, 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 30356 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681715+0000", 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681715+0000", 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681715+0000", 2024-09-16T13:00:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681717+0000", 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.933e-06 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681723+0000", 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9159999999999998e-06 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681726+0000", 2024-09-16T13:00:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3249999999999999e-06 2024-09-16T13:00:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693794+0000", 2024-09-16T13:00:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012067543 2024-09-16T13:00:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693836+0000", 2024-09-16T13:00:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2558999999999997e-05 2024-09-16T13:00:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693873+0000", 2024-09-16T13:00:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7039999999999998e-05 2024-09-16T13:00:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:30357 4.a 4:52996d8f:::rbd_data.11b59afb69bb.000000000000000b:16 [read 3117056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.681730+0000", 2024-09-16T13:00:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4286928269999999, 2024-09-16T13:00:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012243762, 2024-09-16T13:00:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 30357 2024-09-16T13:00:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681730+0000", 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681730+0000", 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681731+0000", 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.06e-07 2024-09-16T13:00:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681733+0000", 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9190000000000002e-06 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681739+0000", 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5150000000000003e-06 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.681744+0000", 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8779999999999997e-06 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693888+0000", 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012143474 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693930+0000", 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2434000000000001e-05 2024-09-16T13:00:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.693974+0000", 2024-09-16T13:00:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3835999999999998e-05 2024-09-16T13:00:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.216 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:05.218 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:05.218 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:05.210+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:05.219 INFO:teuthology.orchestra.run.smithi042.stdout:3411 trunc from 0xa506bf9 to 0xcbebf31 2024-09-16T13:00:05.219 INFO:teuthology.orchestra.run.smithi042.stdout:3412 write 0xaadef13 thru 0xaae0c5d (0x1d4b bytes) 2024-09-16T13:00:05.219 INFO:teuthology.orchestra.run.smithi042.stdout:3413 punch from 0x56330f2 to 0x563d062, (0x9f70 bytes) 2024-09-16T13:00:05.219 INFO:teuthology.orchestra.run.smithi042.stdout:3414 write 0x3fcf143 thru 0x3fdd275 (0xe133 bytes) 2024-09-16T13:00:05.219 INFO:teuthology.orchestra.run.smithi042.stdout:3415 read 0x7a2b1bc thru 0x7a39512 (0xe357 bytes) 2024-09-16T13:00:05.219 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.219 INFO:teuthology.orchestra.run.smithi042.stdout:3418 punch from 0x31e5cb8 to 0x31f3069, (0xd3b1 bytes) 2024-09-16T13:00:05.219 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.219 INFO:teuthology.orchestra.run.smithi042.stdout:3420 read 0x7c1b20 thru 0x7c4d13 (0x31f4 bytes) 2024-09-16T13:00:05.220 INFO:teuthology.orchestra.run.smithi042.stdout:3422 trunc from 0xcbebf31 to 0x9946a7 2024-09-16T13:00:05.220 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T13:00:05.221 INFO:teuthology.orchestra.run.smithi042.stdout:3423 read 0x58bd07 thru 0x593114 (0x740e bytes) 2024-09-16T13:00:05.221 INFO:teuthology.orchestra.run.smithi042.stdout:3424 writesame 0x8090e74 thru 0x8097e81 data_size 0x15(0x700e bytes) 2024-09-16T13:00:05.221 INFO:teuthology.orchestra.run.smithi042.stdout:3427 punch from 0x6f622d6 to 0x6f68750, (0x647a bytes) 2024-09-16T13:00:05.221 INFO:teuthology.orchestra.run.smithi042.stdout:3429 write 0x6f3d061 thru 0x6f4b30d (0xe2ad bytes) 2024-09-16T13:00:05.221 INFO:teuthology.orchestra.run.smithi042.stdout:3430 read 0x4a219a2 thru 0x4a29722 (0x7d81 bytes) 2024-09-16T13:00:05.221 INFO:teuthology.orchestra.run.smithi042.stdout:3432 punch from 0x370d7a7 to 0x370f9f6, (0x224f bytes) 2024-09-16T13:00:05.225 INFO:teuthology.orchestra.run.smithi042.stdout:3433 trunc from 0x8097e82 to 0x1900d22 2024-09-16T13:00:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:3434 punch from 0x12104b9 to 0x121e0a8, (0xdbef bytes) 2024-09-16T13:00:05.243 INFO:teuthology.orchestra.run.smithi042.stdout:3435 punch from 0xaecb4f to 0xaee274, (0x1725 bytes) 2024-09-16T13:00:05.265 INFO:teuthology.orchestra.run.smithi042.stdout:3436 trunc from 0x1900d22 to 0xdf33199 2024-09-16T13:00:05.274 INFO:teuthology.orchestra.run.smithi042.stdout:3437 punch from 0x21ed11b to 0x21faa7d, (0xd962 bytes) 2024-09-16T13:00:05.275 INFO:teuthology.orchestra.run.smithi042.stdout:3439 read 0x8b6231d thru 0x8b71c0c (0xf8f0 bytes) 2024-09-16T13:00:05.275 INFO:teuthology.orchestra.run.smithi042.stdout:3443 write 0x8adad59 thru 0x8ae7ef2 (0xd19a bytes) 2024-09-16T13:00:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:3444 read 0x519c54d thru 0x51a91d7 (0xcc8b bytes) 2024-09-16T13:00:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:3445 write 0xb9ab9e3 thru 0xb9b8748 (0xcd66 bytes) 2024-09-16T13:00:05.288 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.288 INFO:teuthology.orchestra.run.smithi042.stdout:3448 read 0x46ec4cf thru 0x46f6a85 (0xa5b7 bytes) 2024-09-16T13:00:05.288 INFO:teuthology.orchestra.run.smithi042.stdout:3449 read 0x2b60f56 thru 0x2b6cf7d (0xc028 bytes) 2024-09-16T13:00:05.288 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.288 INFO:teuthology.orchestra.run.smithi042.stdout:3451 write 0x73b44de thru 0x73b8869 (0x438c bytes) 2024-09-16T13:00:05.290 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.290 INFO:teuthology.orchestra.run.smithi042.stdout:3453 write 0xda40b8 thru 0xda8525 (0x446e bytes) 2024-09-16T13:00:05.295 INFO:teuthology.orchestra.run.smithi042.stdout:3454 punch from 0x166ac5f to 0x166caa2, (0x1e43 bytes) 2024-09-16T13:00:05.295 INFO:teuthology.orchestra.run.smithi042.stdout:3455 punch from 0xd94083f to 0xd94cdc1, (0xc582 bytes) 2024-09-16T13:00:05.296 INFO:teuthology.orchestra.run.smithi042.stdout:3456 write 0x31fa334 thru 0x32079b7 (0xd684 bytes) 2024-09-16T13:00:05.301 INFO:teuthology.orchestra.run.smithi042.stdout:3458 punch from 0xc7263eb to 0xc733b30, (0xd745 bytes) 2024-09-16T13:00:05.301 INFO:teuthology.orchestra.run.smithi042.stdout:3459 punch from 0x5b1840d to 0x5b25646, (0xd239 bytes) 2024-09-16T13:00:05.302 INFO:teuthology.orchestra.run.smithi042.stdout:3460 writesame 0xc2a68da thru 0xc2b5a6a data_size 0x43(0xf191 bytes) 2024-09-16T13:00:05.305 INFO:teuthology.orchestra.run.smithi042.stdout:3461 write 0x1ff890 thru 0x20f26b (0xf9dc bytes) 2024-09-16T13:00:05.307 INFO:teuthology.orchestra.run.smithi042.stdout:3462 writesame 0x614b4b thru 0x61cf34 data_size 0xb(0x83ea bytes) 2024-09-16T13:00:05.312 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.312 INFO:teuthology.orchestra.run.smithi042.stdout:3464 write 0x4a65c09 thru 0x4a6fa0c (0x9e04 bytes) 2024-09-16T13:00:05.316 INFO:teuthology.orchestra.run.smithi042.stdout:3465 writesame 0xa22c060 thru 0xa231820 data_size 0x1(0x57c1 bytes) 2024-09-16T13:00:05.318 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:05.314+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:05.333 INFO:teuthology.orchestra.run.smithi042.stdout:3467 read 0x568974a thru 0x568f143 (0x59fa bytes) 2024-09-16T13:00:05.334 INFO:teuthology.orchestra.run.smithi042.stdout:3470 read 0x76158a4 thru 0x762338f (0xdaec bytes) 2024-09-16T13:00:05.335 INFO:teuthology.orchestra.run.smithi042.stdout:3472 writesame 0x182f476 thru 0x18330d1 data_size 0x2(0x3c5c bytes) 2024-09-16T13:00:05.343 INFO:teuthology.orchestra.run.smithi042.stdout:3473 read 0x47ca715 thru 0x47cd9fb (0x32e7 bytes) 2024-09-16T13:00:05.362 INFO:teuthology.orchestra.run.smithi042.stdout:3474 trunc from 0xdf33199 to 0x48041e4 2024-09-16T13:00:05.372 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:05.373 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:05.374 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.374 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.382 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:05.394 INFO:teuthology.orchestra.run.smithi042.stdout:3478 writesame 0x8f14ccd thru 0x8f1fed2 data_size 0x1(0xb206 bytes) 2024-09-16T13:00:05.394 INFO:teuthology.orchestra.run.smithi042.stdout:3479 read 0x3e3c3e6 thru 0x3e4b6ec (0xf307 bytes) 2024-09-16T13:00:05.400 INFO:teuthology.orchestra.run.smithi042.stdout:3480 trunc from 0x8f1fed3 to 0x68c44b1 2024-09-16T13:00:05.415 INFO:teuthology.orchestra.run.smithi042.stdout:3481 write 0x1481b5e thru 0x148d5a1 (0xba44 bytes) 2024-09-16T13:00:05.426 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:05.422+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:05.447 INFO:teuthology.orchestra.run.smithi042.stdout:3482 writesame 0xd94a424 thru 0xd953a4c data_size 0x1(0x9629 bytes) 2024-09-16T13:00:05.450 INFO:teuthology.orchestra.run.smithi042.stdout:3483 writesame 0x67f14e7 thru 0x67fd27f data_size 0x3(0xbd99 bytes) 2024-09-16T13:00:05.464 INFO:teuthology.orchestra.run.smithi042.stdout:3485 punch from 0xb6b8361 to 0xb6be72a, (0x63c9 bytes) 2024-09-16T13:00:05.480 INFO:teuthology.orchestra.run.smithi042.stdout:3486 writesame 0xe70f0c6 thru 0xe7195bf data_size 0x3(0xa4fa bytes) 2024-09-16T13:00:05.483 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.483 INFO:teuthology.orchestra.run.smithi042.stdout:3488 read 0x8bb1c83 thru 0x8bb2f12 (0x1290 bytes) 2024-09-16T13:00:05.484 INFO:teuthology.orchestra.run.smithi042.stdout:3489 read 0x8490d28 thru 0x849bb01 (0xadda bytes) 2024-09-16T13:00:05.485 INFO:teuthology.orchestra.run.smithi042.stdout:3490 write 0xc9fc53 thru 0xcac7d7 (0xcb85 bytes) 2024-09-16T13:00:05.487 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:05.487 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.488 INFO:teuthology.orchestra.run.smithi042.stdout:3492 write 0x7e07ac1 thru 0x7e0fa6e (0x7fae bytes) 2024-09-16T13:00:05.491 INFO:teuthology.orchestra.run.smithi042.stdout:3493 read 0xc12dc42 thru 0xc139875 (0xbc34 bytes) 2024-09-16T13:00:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:3495 read 0xd3c4207 thru 0xd3cdbde (0x99d8 bytes) 2024-09-16T13:00:05.492 INFO:teuthology.orchestra.run.smithi042.stdout:3497 writesame 0x57c18a9 thru 0x57ce9eb data_size 0x7(0xd143 bytes) 2024-09-16T13:00:05.496 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.503 INFO:teuthology.orchestra.run.smithi042.stdout:3500 read 0x391883f thru 0x391d07c (0x483e bytes) 2024-09-16T13:00:05.503 INFO:teuthology.orchestra.run.smithi042.stdout:3502 writesame 0x39e745d thru 0x39f05fc data_size 0x50(0x91a0 bytes) 2024-09-16T13:00:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:3506 punch from 0x924eb31 to 0x9254ffb, (0x64ca bytes) 2024-09-16T13:00:05.506 INFO:teuthology.orchestra.run.smithi042.stdout:3507 read 0xcad7cef thru 0xcae76fb (0xfa0d bytes) 2024-09-16T13:00:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:3508 trunc from 0xe7195c0 to 0xde8ea4d 2024-09-16T13:00:05.520 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:05.514+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:05.522 INFO:teuthology.orchestra.run.smithi042.stdout:3509 read 0x141faf7 thru 0x142d6c6 (0xdbd0 bytes) 2024-09-16T13:00:05.523 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.523 INFO:teuthology.orchestra.run.smithi042.stdout:3511 writesame 0x705ace6 thru 0x706050e data_size 0x1(0x5829 bytes) 2024-09-16T13:00:05.544 INFO:teuthology.orchestra.run.smithi042.stdout:3512 punch from 0x6b63c8 to 0x6bf602, (0x923a bytes) 2024-09-16T13:00:05.544 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.544 INFO:teuthology.orchestra.run.smithi042.stdout:3515 writesame 0x351464c thru 0x351b180 data_size 0xb(0x6b35 bytes) 2024-09-16T13:00:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:3517 read 0x8a9fd9c thru 0x8aa4611 (0x4876 bytes) 2024-09-16T13:00:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:3518 read 0x8492bad thru 0x849459e (0x19f2 bytes) 2024-09-16T13:00:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:3519 punch from 0x941ba7f to 0x94251da, (0x975b bytes) 2024-09-16T13:00:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:3521 writesame 0x449d9a9 thru 0x44ad13f data_size 0x1(0xf797 bytes) 2024-09-16T13:00:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:3522 read 0x7dc6a28 thru 0x7dd2f67 (0xc540 bytes) 2024-09-16T13:00:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.595 INFO:teuthology.orchestra.run.smithi042.stdout:3524 writesame 0x9da8d6b thru 0x9db63ab data_size 0x2f(0xd641 bytes) 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.043284166999996, 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:00:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374548+0000", 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.218457583000003, 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010809278, 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44 2024-09-16T13:00:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374549+0000", 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.037e-06 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374711+0000", 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016226599999999999 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374734+0000", 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2895999999999999e-05 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374746+0000", 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.221e-05 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.380942+0000", 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061959390000000001 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381063+0000", 2024-09-16T13:00:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121274 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381255+0000", 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019110500000000001 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384487+0000", 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032327979999999998 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385296+0000", 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080888499999999996 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385332+0000", 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5774e-05 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385357+0000", 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5094000000000001e-05 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.218246691000004, 2024-09-16T13:00:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010976025, 2024-09-16T13:00:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 45 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374760+0000", 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.13e-06 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374769+0000", 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5699999999999999e-06 2024-09-16T13:00:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374819+0000", 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9882000000000001e-05 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374828+0000", 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382329+0000", 2024-09-16T13:00:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075009009999999999 2024-09-16T13:00:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382403+0000", 2024-09-16T13:00:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4208999999999994e-05 2024-09-16T13:00:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382514+0000", 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111276 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385533+0000", 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030190080000000001 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385564+0000", 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0926000000000002e-05 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385666+0000", 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101791 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385735+0000", 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8541999999999995e-05 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.218167120000004, 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:00:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.218119864000002, 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:00:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:00:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.217916927999994, 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:00:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:00:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.217885666000001, 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:00:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.217849904000005, 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:00:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:00:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.217655273999995, 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:00:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.217608428999995, 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:00:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919457+0000", 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.673548388, 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010623719, 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 69 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919458+0000", 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.474e-06 2024-09-16T13:00:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919621+0000", 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016234800000000001 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919641+0000", 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0336999999999999e-05 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919656+0000", 2024-09-16T13:00:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4554e-05 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924273+0000", 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046169610000000002 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924411+0000", 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138286 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924627+0000", 2024-09-16T13:00:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021597399999999999 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928794+0000", 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004167268 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929983+0000", 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011884409999999999 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930039+0000", 2024-09-16T13:00:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6363000000000002e-05 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930081+0000", 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1712999999999999e-05 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.673332258000002, 2024-09-16T13:00:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:00:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.673278350999993, 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:00:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:00:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.673236046, 2024-09-16T13:00:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:00:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:00:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.673077621000004, 2024-09-16T13:00:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:00:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:00:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.67302814, 2024-09-16T13:00:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:00:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:00:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.672897570000003, 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:00:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:00:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:00:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.672716057000002, 2024-09-16T13:00:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:00:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 77.86218839, 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:00:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:00:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:00:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:00:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:00:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:00:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:00:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 77.862039436000003, 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:00:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:05.720 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:05.720 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:05.722 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:05.714+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:05.722 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:05.724 INFO:teuthology.orchestra.run.smithi042.stdout:3525 write 0x8c44809 thru 0x8c4dbcf (0x93c7 bytes) 2024-09-16T13:00:05.724 INFO:teuthology.orchestra.run.smithi042.stdout:3527 writesame 0xcd3291d thru 0xcd37a29 data_size 0x1(0x510d bytes) 2024-09-16T13:00:05.724 INFO:teuthology.orchestra.run.smithi042.stdout:3528 writesame 0x1c372b2 thru 0x1c37a2b data_size 0x16(0x77a bytes) 2024-09-16T13:00:05.724 INFO:teuthology.orchestra.run.smithi042.stdout:3529 writesame 0x3258e27 thru 0x326353c data_size 0x3(0xa716 bytes) 2024-09-16T13:00:05.724 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.724 INFO:teuthology.orchestra.run.smithi042.stdout:3533 write 0x9f1b354 thru 0x9f20c59 (0x5906 bytes) 2024-09-16T13:00:05.724 INFO:teuthology.orchestra.run.smithi042.stdout:3534 trunc from 0xde8ea4d to 0xb84e7d7 2024-09-16T13:00:05.724 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.724 INFO:teuthology.orchestra.run.smithi042.stdout:3537 trunc from 0xb84e7d7 to 0x98bccf2 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:3539 read 0x5b4c7e9 thru 0x5b58a57 (0xc26f bytes) 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:3542 write 0x15e8a97 thru 0x15f4aec (0xc056 bytes) 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:3543 punch from 0x14e076 to 0x15b25e, (0xd1e8 bytes) 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:3544 read 0x46b1342 thru 0x46bb492 (0xa151 bytes) 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:3546 punch from 0x8425260 to 0x84337fd, (0xe59d bytes) 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:3549 trunc from 0x98bccf2 to 0x75113ff 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:3550 read 0x59dd001 thru 0x59e0325 (0x3325 bytes) 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:3551 writesame 0xa10182 thru 0xa1e8b7 data_size 0xf(0xe736 bytes) 2024-09-16T13:00:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:3554 write 0x504bff3 thru 0x5054d73 (0x8d81 bytes) 2024-09-16T13:00:05.726 INFO:teuthology.orchestra.run.smithi042.stdout:3557 writesame 0x2d13450 thru 0x2d13d35 data_size 0x11(0x8e6 bytes) 2024-09-16T13:00:05.729 INFO:teuthology.orchestra.run.smithi042.stdout:3559 read 0x3640f49 thru 0x364173c (0x7f4 bytes) 2024-09-16T13:00:05.730 INFO:teuthology.orchestra.run.smithi042.stdout:3561 read 0x29ae8f5 thru 0x29b1a25 (0x3131 bytes) 2024-09-16T13:00:05.730 INFO:teuthology.orchestra.run.smithi042.stdout:3562 read 0x2013095 thru 0x20203ab (0xd317 bytes) 2024-09-16T13:00:05.744 INFO:teuthology.orchestra.run.smithi042.stdout:3564 trunc from 0x75113ff to 0xd531add 2024-09-16T13:00:05.753 INFO:teuthology.orchestra.run.smithi042.stdout:3566 read 0xbab645 thru 0xbb8e53 (0xd80f bytes) 2024-09-16T13:00:05.753 INFO:teuthology.orchestra.run.smithi042.stdout:3567 writesame 0x3a1190b thru 0x3a1414c data_size 0x1(0x2842 bytes) 2024-09-16T13:00:05.761 INFO:teuthology.orchestra.run.smithi042.stdout:3568 read 0x68e8729 thru 0x68f46f4 (0xbfcc bytes) 2024-09-16T13:00:05.781 INFO:teuthology.orchestra.run.smithi042.stdout:3569 trunc from 0xd531add to 0x1254dd1 2024-09-16T13:00:05.814 INFO:teuthology.orchestra.run.smithi042.stdout:3571 read 0x3031b8 thru 0x30c4a6 (0x92ef bytes) 2024-09-16T13:00:05.815 INFO:teuthology.orchestra.run.smithi042.stdout:3572 punch from 0x56a707 to 0x572c5d, (0x8556 bytes) 2024-09-16T13:00:05.815 INFO:teuthology.orchestra.run.smithi042.stdout:3573 write 0x78c755 thru 0x78efbf (0x286b bytes) 2024-09-16T13:00:05.819 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.819 INFO:teuthology.orchestra.run.smithi042.stdout:3576 write 0x4083c4 thru 0x416a5b (0xe698 bytes) 2024-09-16T13:00:05.822 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:05.818+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:05.823 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.835 INFO:teuthology.orchestra.run.smithi042.stdout:3579 write 0x212ce9e thru 0x2134d65 (0x7ec8 bytes) 2024-09-16T13:00:05.842 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:05.842 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:05.842 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:05.842 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:05.853 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:05.855 INFO:teuthology.orchestra.run.smithi042.stdout:3581 write 0x3cbe336 thru 0x3cc470a (0x63d5 bytes) 2024-09-16T13:00:05.879 INFO:teuthology.orchestra.run.smithi042.stdout:3582 writesame 0xa04705e thru 0xa05166d data_size 0x8(0xa610 bytes) 2024-09-16T13:00:05.880 INFO:teuthology.orchestra.run.smithi042.stdout:3583 write 0xccfd60 thru 0xcd348f (0x3730 bytes) 2024-09-16T13:00:05.882 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.882 INFO:teuthology.orchestra.run.smithi042.stdout:3585 write 0x9cd17db thru 0x9cdf7ca (0xdff0 bytes) 2024-09-16T13:00:05.887 INFO:teuthology.orchestra.run.smithi042.stdout:3586 read 0x7c2ab5b thru 0x7c37040 (0xc4e6 bytes) 2024-09-16T13:00:05.887 INFO:teuthology.orchestra.run.smithi042.stdout:3588 write 0x982a56 thru 0x98dddb (0xb386 bytes) 2024-09-16T13:00:05.890 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.891 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:05.898 INFO:teuthology.orchestra.run.smithi042.stdout:3591 trunc from 0xa05166e to 0x5c6a118 2024-09-16T13:00:05.916 INFO:teuthology.orchestra.run.smithi042.stdout:3594 trunc from 0x5c6a118 to 0x341d135 2024-09-16T13:00:05.923 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:05.918+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:05.930 INFO:teuthology.orchestra.run.smithi042.stdout:3595 read 0x2ff7dec thru 0x3005282 (0xd497 bytes) 2024-09-16T13:00:05.931 INFO:teuthology.orchestra.run.smithi042.stdout:3596 write 0x1fa9a2c thru 0x1faa9f6 (0xfcb bytes) 2024-09-16T13:00:05.942 INFO:teuthology.orchestra.run.smithi042.stdout:3597 trunc from 0x341d135 to 0x7108ed 2024-09-16T13:00:05.968 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:05.968 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:05.968 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:05.968 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:05.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:05.986 INFO:teuthology.orchestra.run.smithi042.stdout:3598 write 0x5a560e9 thru 0x5a65726 (0xf63e bytes) 2024-09-16T13:00:05.992 INFO:teuthology.orchestra.run.smithi042.stdout:3599 read 0x1d4ca36 thru 0x1d4eacb (0x2096 bytes) 2024-09-16T13:00:05.992 INFO:teuthology.orchestra.run.smithi042.stdout:3600 punch from 0xb5dd50 to 0xb69186, (0xb436 bytes) 2024-09-16T13:00:05.992 INFO:teuthology.orchestra.run.smithi042.stdout:3601 read 0x1442203 thru 0x144ea73 (0xc871 bytes) 2024-09-16T13:00:06.015 INFO:teuthology.orchestra.run.smithi042.stdout:3602 write 0xc45b7db thru 0xc45bac1 (0x2e7 bytes) 2024-09-16T13:00:06.021 INFO:teuthology.orchestra.run.smithi042.stdout:3604 writesame 0xb4b89f0 thru 0xb4c2578 data_size 0x1(0x9b89 bytes) 2024-09-16T13:00:06.024 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:06.018+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:06.055 INFO:teuthology.orchestra.run.smithi042.stdout:3605 trunc from 0xc45bac2 to 0xbe5ba1b 2024-09-16T13:00:06.068 INFO:teuthology.orchestra.run.smithi042.stdout:3606 trunc from 0xbe5ba1b to 0xb0eb69a 2024-09-16T13:00:06.080 INFO:teuthology.orchestra.run.smithi042.stdout:3607 write 0x33ce14d thru 0x33cff63 (0x1e17 bytes) 2024-09-16T13:00:06.085 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.085 INFO:teuthology.orchestra.run.smithi042.stdout:3611 read 0xa85722 thru 0xa8ec38 (0x9517 bytes) 2024-09-16T13:00:06.086 INFO:teuthology.orchestra.run.smithi042.stdout:3612 punch from 0x759f49b to 0x75a61fa, (0x6d5f bytes) 2024-09-16T13:00:06.086 INFO:teuthology.orchestra.run.smithi042.stdout:3614 writesame 0x831ff5b thru 0x8328597 data_size 0x57(0x863d bytes) 2024-09-16T13:00:06.091 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:06.091 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:636 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.472199+0000", 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 48.614121421, 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027791489999999999, 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 636 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472202+0000", 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.01e-06 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472218+0000", 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5724000000000001e-05 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472261+0000", 2024-09-16T13:00:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2858999999999997e-05 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472277+0000", 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6297999999999999e-05 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472397+0000", 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012026199999999999 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472563+0000", 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016614900000000001 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.480681+0000", 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081177869999999996 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499671+0000", 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018990289 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499756+0000", 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4511000000000005e-05 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499858+0000", 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102495 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499991+0000", 2024-09-16T13:00:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132106 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3577 4.1e 4:7a8a5a14:::rbd_data.11b5c378985b.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.100645+0000", 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.985674969000002, 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027798529999999998, 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3577 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T13:00:06.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100649+0000", 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1769999999999998e-06 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100659+0000", 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0407e-05 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100689+0000", 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0290999999999998e-05 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100701+0000", 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1960999999999999e-05 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100800+0000", 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8350000000000005e-05 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.101027+0000", 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000227736 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.111830+0000", 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010802735000000001 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128285+0000", 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016454389999999999 2024-09-16T13:00:06.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128338+0000", 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3112000000000002e-05 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128398+0000", 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0136999999999997e-05 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128444+0000", 2024-09-16T13:00:06.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6233999999999998e-05 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5229 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.838222+0000", 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.248098642000002, 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049016400000000002, 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5229 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838225+0000", 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0340000000000001e-06 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838239+0000", 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4004e-05 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838280+0000", 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0738000000000002e-05 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838294+0000", 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.482e-05 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838415+0000", 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012084 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838570+0000", 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000155162 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.850428+0000", 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011857826 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887135+0000", 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036706938000000001 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887149+0000", 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3421000000000001e-05 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887202+0000", 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3427000000000002e-05 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887238+0000", 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6189999999999997e-05 2024-09-16T13:00:06.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5396 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.222043+0000", 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.864277518999998, 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035598629999999999, 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5396 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222045+0000", 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9580000000000001e-06 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222055+0000", 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9310000000000003e-06 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222082+0000", 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6882000000000001e-05 2024-09-16T13:00:06.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222092+0000", 2024-09-16T13:00:06.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8910000000000007e-06 2024-09-16T13:00:06.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222166+0000", 2024-09-16T13:00:06.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4437999999999994e-05 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222264+0000", 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7748e-05 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.228318+0000", 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060540109999999998 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257411+0000", 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029093622999999999 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257500+0000", 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8341e-05 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257580+0000", 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-16T13:00:06.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257642+0000", 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1073000000000003e-05 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:18283 4.1e 4:7c33f2a6:::rbd_data.11b51a3eb68b.000000000000002b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:56.586760+0000", 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4995599909999999, 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.031869502000000001, 2024-09-16T13:00:06.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:06.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18283 2024-09-16T13:00:06.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.117 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586760+0000", 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586760+0000", 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586763+0000", 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0170000000000001e-06 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586777+0000", 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3692999999999999e-05 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586817+0000", 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9796999999999999e-05 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586856+0000", 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8829999999999999e-05 2024-09-16T13:00:06.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586975+0000", 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119083 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.587231+0000", 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025600999999999998 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:06.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.596243+0000", 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090123749999999996 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.618539+0000", 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022296137000000001 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.618548+0000", 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9770000000000006e-06 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.618588+0000", 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9681999999999997e-05 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.618630+0000", 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1900999999999997e-05 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.122 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:27166 4.6 4:67a7dc0b:::rbd_data.11b59afb69bb.0000000000000006:16 [read 2195456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.567014+0000", 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.5193065150000002, 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020993722999999999, 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 27166 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567014+0000", 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567014+0000", 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567014+0000", 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567017+0000", 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3819999999999998e-06 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567024+0000", 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9480000000000002e-06 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567028+0000", 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9700000000000001e-06 2024-09-16T13:00:06.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.582236+0000", 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015208172000000001 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.582312+0000", 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5903999999999997e-05 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.588008+0000", 2024-09-16T13:00:06.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056959089999999999 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32029 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3543040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.736668+0000", 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3496529410000004, 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022092409, 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32029 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736668+0000", 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736668+0000", 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736668+0000", 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3e-07 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736671+0000", 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5069999999999999e-06 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736678+0000", 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4200000000000001e-06 2024-09-16T13:00:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736682+0000", 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8840000000000004e-06 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.755361+0000", 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018678555999999999 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.755484+0000", 2024-09-16T13:00:06.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306599999999999 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.758760+0000", 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003276246 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32033 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.736733+0000", 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3495879029999998, 2024-09-16T13:00:06.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019686207000000001, 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32033 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736733+0000", 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736733+0000", 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736733+0000", 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6100000000000002e-07 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736735+0000", 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.973e-06 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736741+0000", 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736746+0000", 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2490000000000001e-06 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756236+0000", 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019490176000000001 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756337+0000", 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100455 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756419+0000", 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2103999999999999e-05 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.135 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32063 4.6 4:67a7dc0b:::rbd_data.11b59afb69bb.0000000000000006:16 [read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737067+0000", 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3492531720000001, 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019817906999999999, 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.136 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32063 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737067+0000", 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737067+0000", 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737068+0000", 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737070+0000", 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2649999999999999e-06 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737076+0000", 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7610000000000004e-06 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737079+0000", 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6650000000000001e-06 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756739+0000", 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019659421999999999 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756822+0000", 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3013e-05 2024-09-16T13:00:06.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756885+0000", 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3282999999999997e-05 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32064 4.6 4:67a7dc0b:::rbd_data.11b59afb69bb.0000000000000006:16 [read 3633152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737083+0000", 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3492370119999997, 2024-09-16T13:00:06.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019903495, 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32064 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737083+0000", 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737083+0000", 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737084+0000", 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737086+0000", 2024-09-16T13:00:06.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2409999999999998e-06 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737092+0000", 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8109999999999999e-06 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737095+0000", 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5379999999999998e-06 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756913+0000", 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019817952999999999 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756954+0000", 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0676000000000002e-05 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756987+0000", 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2898000000000002e-05 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.144 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32065 4.6 4:67a7dc0b:::rbd_data.11b59afb69bb.0000000000000006:16 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737099+0000", 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3492210140000003, 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020025591999999998, 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32065 2024-09-16T13:00:06.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737099+0000", 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737099+0000", 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737100+0000", 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737102+0000", 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0619999999999999e-06 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737108+0000", 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6679999999999998e-06 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737111+0000", 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4010000000000001e-06 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757000+0000", 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019888922999999999 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757058+0000", 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7595e-05 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757125+0000", 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7442999999999998e-05 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32093 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3547136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.149 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737564+0000", 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3487564120000002, 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019709436, 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32093 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737564+0000", 2024-09-16T13:00:06.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737564+0000", 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737565+0000", 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.47e-07 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737570+0000", 2024-09-16T13:00:06.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8389999999999997e-06 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737583+0000", 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2843000000000001e-05 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737589+0000", 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0800000000000002e-06 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757146+0000", 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019557261999999999 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757192+0000", 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5754000000000003e-05 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757273+0000", 2024-09-16T13:00:06.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1811000000000007e-05 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32094 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737595+0000", 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3487255119999997, 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019827239999999999, 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32094 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737595+0000", 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737595+0000", 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737595+0000", 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737598+0000", 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8159999999999998e-06 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737605+0000", 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3810000000000004e-06 2024-09-16T13:00:06.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737609+0000", 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0890000000000002e-06 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757295+0000", 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019686454999999999 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757346+0000", 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0952999999999998e-05 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757422+0000", 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6015999999999999e-05 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32095 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3604480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737613+0000", 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3487075040000001, 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019949204000000002, 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32095 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737613+0000", 2024-09-16T13:00:06.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737613+0000", 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737614+0000", 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7899999999999998e-07 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737616+0000", 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9080000000000002e-06 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737622+0000", 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1920000000000002e-06 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737626+0000", 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0199999999999996e-06 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757453+0000", 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019827506000000002 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757515+0000", 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1254999999999995e-05 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757562+0000", 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7644e-05 2024-09-16T13:00:06.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32096 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737630+0000", 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3486904199999996, 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020104349000000001, 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.163 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32096 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737630+0000", 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737630+0000", 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737630+0000", 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737632+0000", 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737638+0000", 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7810000000000002e-06 2024-09-16T13:00:06.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737642+0000", 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.692e-06 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757580+0000", 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019937723000000001 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757656+0000", 2024-09-16T13:00:06.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.606e-05 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757734+0000", 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8658999999999993e-05 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32097 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3661824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737646+0000", 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3486745249999998, 2024-09-16T13:00:06.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020224474999999999, 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32097 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737646+0000", 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737646+0000", 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737646+0000", 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8999999999999997e-07 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737648+0000", 2024-09-16T13:00:06.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.979e-06 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737655+0000", 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6959999999999999e-06 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737659+0000", 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9670000000000003e-06 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757757+0000", 2024-09-16T13:00:06.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020098443000000001 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757819+0000", 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1369000000000001e-05 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757870+0000", 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1530999999999998e-05 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42592 4.6 4:611ecd9e:::rbd_data.11b52dde2e.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.300454+0000", 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7858666989999996, 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025747084999999999, 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42592 2024-09-16T13:00:06.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300454+0000", 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300454+0000", 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300455+0000", 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4720000000000001e-06 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300462+0000", 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0520000000000004e-06 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300483+0000", 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0466e-05 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300491+0000", 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5380000000000002e-06 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300553+0000", 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2150999999999995e-05 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300655+0000", 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010206599999999999 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.304922+0000", 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042666730000000003 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.326046+0000", 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021124079 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.326072+0000", 2024-09-16T13:00:06.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6213000000000001e-05 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.326146+0000", 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3267000000000002e-05 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.326201+0000", 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5108000000000002e-05 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42607 4.3 4:c3aab101:::rbd_data.11b52dde2e.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.345306+0000", 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.741014871, 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036778706000000001, 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42607 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.178 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345306+0000", 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345306+0000", 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345307+0000", 2024-09-16T13:00:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4300000000000001e-06 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345314+0000", 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3000000000000004e-06 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345334+0000", 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9794000000000001e-05 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345342+0000", 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9519999999999994e-06 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345398+0000", 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6347000000000001e-05 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345471+0000", 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2399999999999998e-05 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.352464+0000", 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069934630000000001 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.381945+0000", 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029481120999999999 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.381962+0000", 2024-09-16T13:00:06.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7030000000000001e-05 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.382034+0000", 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1693999999999995e-05 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.382084+0000", 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0175e-05 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44624 4.6 4:611ecd9e:::rbd_data.11b52dde2e.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:05.550726+0000", 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.53559447900000001, 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041671303999999999, 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44624 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550726+0000", 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550726+0000", 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550729+0000", 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.49e-06 2024-09-16T13:00:06.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550746+0000", 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.632e-05 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550788+0000", 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2549999999999997e-05 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550804+0000", 2024-09-16T13:00:06.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6140000000000001e-05 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550895+0000", 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0585000000000001e-05 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550982+0000", 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6916999999999995e-05 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:06.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.558031+0000", 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070490539999999999 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.592241+0000", 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.034209943999999999 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.592296+0000", 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4882000000000001e-05 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.592354+0000", 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8295999999999999e-05 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.592397+0000", 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3126e-05 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.189 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44860 4.6 4:611ecd9e:::rbd_data.11b52dde2e.0000000000000003:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:06.021827+0000", 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.064493588000000004, 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030066649000000001, 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44860 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:06.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021827+0000", 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021827+0000", 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021829+0000", 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.543e-06 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021839+0000", 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7370000000000001e-06 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021864+0000", 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4850000000000001e-05 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021878+0000", 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3935e-05 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021962+0000", 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4201000000000002e-05 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.022043+0000", 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1241000000000004e-05 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.027176+0000", 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051329009999999996 2024-09-16T13:00:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.051817+0000", 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024640815999999999 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.051825+0000", 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1580000000000005e-06 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.051867+0000", 2024-09-16T13:00:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1499999999999999e-05 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.051894+0000", 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6767e-05 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:06.195 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:06.196 INFO:teuthology.orchestra.run.smithi042.stdout:3615 writesame 0x74806e2 thru 0x748ffae data_size 0x9(0xf8cd bytes) 2024-09-16T13:00:06.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:06.198 INFO:teuthology.orchestra.run.smithi042.stdout:3617 writesame 0xaefc7a8 thru 0xaefe873 data_size 0x2(0x20cc bytes) 2024-09-16T13:00:06.198 INFO:teuthology.orchestra.run.smithi042.stdout:3618 write 0x5ddef8c thru 0x5de9395 (0xa40a bytes) 2024-09-16T13:00:06.198 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.198 INFO:teuthology.orchestra.run.smithi042.stdout:3621 punch from 0x87ea462 to 0x87f9b42, (0xf6e0 bytes) 2024-09-16T13:00:06.198 INFO:teuthology.orchestra.run.smithi042.stdout:3623 write 0x1c9f39c thru 0x1cad2aa (0xdf0f bytes) 2024-09-16T13:00:06.198 INFO:teuthology.orchestra.run.smithi042.stdout:3625 writesame 0xbdaafd3 thru 0xbdaefc0 data_size 0xe(0x3fee bytes) 2024-09-16T13:00:06.198 INFO:teuthology.orchestra.run.smithi042.stdout:3626 write 0xe4070cb thru 0xe40b33f (0x4275 bytes) 2024-09-16T13:00:06.199 INFO:teuthology.orchestra.run.smithi042.stdout:3627 write 0xadf4942 thru 0xadfe0e1 (0x97a0 bytes) 2024-09-16T13:00:06.199 INFO:teuthology.orchestra.run.smithi042.stdout:3628 writesame 0x25289fc thru 0x25368a3 data_size 0xc8(0xdea8 bytes) 2024-09-16T13:00:06.199 INFO:teuthology.orchestra.run.smithi042.stdout:3629 punch from 0xd67497 to 0xd76cf1, (0xf85a bytes) 2024-09-16T13:00:06.199 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.199 INFO:teuthology.orchestra.run.smithi042.stdout:3631 punch from 0xd2e247c to 0xd2f19f1, (0xf575 bytes) 2024-09-16T13:00:06.199 INFO:teuthology.orchestra.run.smithi042.stdout:3632 trunc from 0xe40b340 to 0x6b9cb35 2024-09-16T13:00:06.199 INFO:teuthology.orchestra.run.smithi042.stdout:3634 write 0x49425bc thru 0x4948c0b (0x6650 bytes) 2024-09-16T13:00:06.200 INFO:teuthology.orchestra.run.smithi042.stdout:3635 trunc from 0x6b9cb35 to 0x9ef19fe 2024-09-16T13:00:06.200 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:06.194+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:06.210 INFO:teuthology.orchestra.run.smithi042.stdout:3636 read 0x8f6664e thru 0x8f6ec7c (0x862f bytes) 2024-09-16T13:00:06.222 INFO:teuthology.orchestra.run.smithi042.stdout:3638 write 0xa4da4d4 thru 0xa4e955f (0xf08c bytes) 2024-09-16T13:00:06.228 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.228 INFO:teuthology.orchestra.run.smithi042.stdout:3640 punch from 0x322190d to 0x322c08f, (0xa782 bytes) 2024-09-16T13:00:06.228 INFO:teuthology.orchestra.run.smithi042.stdout:3641 read 0x60a7ce5 thru 0x60b02ed (0x8609 bytes) 2024-09-16T13:00:06.247 INFO:teuthology.orchestra.run.smithi042.stdout:3642 write 0xd905986 thru 0xd90fa19 (0xa094 bytes) 2024-09-16T13:00:06.261 INFO:teuthology.orchestra.run.smithi042.stdout:3644 trunc from 0xd90fa1a to 0x7d8f3fe 2024-09-16T13:00:06.276 INFO:teuthology.orchestra.run.smithi042.stdout:3645 read 0x70c7822 thru 0x70cf3bd (0x7b9c bytes) 2024-09-16T13:00:06.284 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:06.292 INFO:teuthology.orchestra.run.smithi042.stdout:3646 writesame 0xb8e31d9 thru 0xb8e6335 data_size 0x1(0x315d bytes) 2024-09-16T13:00:06.294 INFO:teuthology.orchestra.run.smithi042.stdout:3647 write 0xacdcbcc thru 0xace02a8 (0x36dd bytes) 2024-09-16T13:00:06.300 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:06.291+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:06.310 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:06.310 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:06.313 INFO:teuthology.orchestra.run.smithi042.stdout:3648 trunc from 0xb8e6336 to 0x3dbf4e5 2024-09-16T13:00:06.322 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:06.327 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.327 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.336 INFO:teuthology.orchestra.run.smithi042.stdout:3651 writesame 0x43fc3f5 thru 0x43ffccd data_size 0x21(0x38d9 bytes) 2024-09-16T13:00:06.336 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.337 INFO:teuthology.orchestra.run.smithi042.stdout:3654 punch from 0x29717cb to 0x297ddbe, (0xc5f3 bytes) 2024-09-16T13:00:06.357 INFO:teuthology.orchestra.run.smithi042.stdout:3655 write 0xa76ae06 thru 0xa777ee1 (0xd0dc bytes) 2024-09-16T13:00:06.359 INFO:teuthology.orchestra.run.smithi042.stdout:3656 punch from 0x49bc479 to 0x49c4362, (0x7ee9 bytes) 2024-09-16T13:00:06.359 INFO:teuthology.orchestra.run.smithi042.stdout:3657 read 0x940cd4b thru 0x9414399 (0x764f bytes) 2024-09-16T13:00:06.367 INFO:teuthology.orchestra.run.smithi042.stdout:3658 trunc from 0xa777ee2 to 0x6607970 2024-09-16T13:00:06.381 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.381 INFO:teuthology.orchestra.run.smithi042.stdout:3660 read 0x40de98a thru 0x40e2e05 (0x447c bytes) 2024-09-16T13:00:06.381 INFO:teuthology.orchestra.run.smithi042.stdout:3661 write 0x5cf17a8 thru 0x5cfcb99 (0xb3f2 bytes) 2024-09-16T13:00:06.384 INFO:teuthology.orchestra.run.smithi042.stdout:3662 writesame 0x6267d8c thru 0x6272fcd data_size 0x1(0xb242 bytes) 2024-09-16T13:00:06.401 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:06.395+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:06.426 INFO:teuthology.orchestra.run.smithi042.stdout:3663 trunc from 0x6607970 to 0xc7dbe29 2024-09-16T13:00:06.429 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:06.430 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:06.430 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:06.430 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:06.430 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:06.430 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:06.434 INFO:teuthology.orchestra.run.smithi042.stdout:3666 read 0x35bf6 thru 0x4297a (0xcd85 bytes) 2024-09-16T13:00:06.440 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:06.444 INFO:teuthology.orchestra.run.smithi042.stdout:3667 trunc from 0xc7dbe29 to 0x6f34b9b 2024-09-16T13:00:06.454 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:06.457 INFO:teuthology.orchestra.run.smithi042.stdout:3668 write 0x5f36723 thru 0x5f39376 (0x2c54 bytes) 2024-09-16T13:00:06.480 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:06.480 INFO:teuthology.orchestra.run.smithi042.stdout:3670 write 0xda64183 thru 0xda6a2d9 (0x6157 bytes) 2024-09-16T13:00:06.482 INFO:teuthology.orchestra.run.smithi042.stdout:3671 read 0xb6cfd1d thru 0xb6dbf82 (0xc266 bytes) 2024-09-16T13:00:06.482 INFO:teuthology.orchestra.run.smithi042.stdout:3672 read 0x34fa06d thru 0x3504594 (0xa528 bytes) 2024-09-16T13:00:06.483 INFO:teuthology.orchestra.run.smithi042.stdout:3676 punch from 0xa12029b to 0xa12b03d, (0xada2 bytes) 2024-09-16T13:00:06.483 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.483 INFO:teuthology.orchestra.run.smithi042.stdout:3678 read 0x16e88f1 thru 0x16f5c0c (0xd31c bytes) 2024-09-16T13:00:06.494 INFO:teuthology.orchestra.run.smithi042.stdout:3679 write 0xed8c81a thru 0xed9c3d3 (0xfbba bytes) 2024-09-16T13:00:06.496 INFO:teuthology.orchestra.run.smithi042.stdout:3680 writesame 0x1df1c35 thru 0x1df7679 data_size 0x1(0x5a45 bytes) 2024-09-16T13:00:06.502 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:06.498+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:06.513 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:06.514 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.514 INFO:teuthology.orchestra.run.smithi042.stdout:3682 read 0x3dc2b7e thru 0x3dc740d (0x4890 bytes) 2024-09-16T13:00:06.515 INFO:teuthology.orchestra.run.smithi042.stdout:3684 writesame 0x9bfbc77 thru 0x9c0b7fd data_size 0x1(0xfb87 bytes) 2024-09-16T13:00:06.540 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:00:06.540 INFO:teuthology.orchestra.run.smithi042.stdout:3685 write 0xc424045 thru 0xc426d91 (0x2d4d bytes) 2024-09-16T13:00:06.560 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:06.560 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:06.561 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:06.562 INFO:teuthology.orchestra.run.smithi042.stdout:3686 trunc from 0xed9c3d4 to 0x1e72d8e 2024-09-16T13:00:06.573 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:06.589 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:06.603 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:06.598+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:06.603 INFO:teuthology.orchestra.run.smithi042.stdout:3687 write 0x71b7826 thru 0x71c69b0 (0xf18b bytes) 2024-09-16T13:00:06.616 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:00:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:06.624 INFO:teuthology.orchestra.run.smithi042.stdout:3688 write 0xdac43c6 thru 0xdad3ea8 (0xfae3 bytes) 2024-09-16T13:00:06.634 INFO:teuthology.orchestra.run.smithi042.stdout:3689 trunc from 0xdad3ea9 to 0x9c35aa8 2024-09-16T13:00:06.648 INFO:teuthology.orchestra.run.smithi042.stdout:3690 trunc from 0x9c35aa8 to 0xa6a85cb 2024-09-16T13:00:06.657 INFO:teuthology.orchestra.run.smithi042.stdout:3692 read 0x3a5db44 thru 0x3a5f6ca (0x1b87 bytes) 2024-09-16T13:00:06.657 INFO:teuthology.orchestra.run.smithi042.stdout:3695 read 0x45d4d12 thru 0x45dfe12 (0xb101 bytes) 2024-09-16T13:00:06.659 INFO:teuthology.orchestra.run.smithi042.stdout:3696 trunc from 0xa6a85cb to 0xa0f72e1 2024-09-16T13:00:06.676 INFO:teuthology.orchestra.run.smithi042.stdout:3698 trunc from 0xa0f72e1 to 0xc6d97cf 2024-09-16T13:00:06.688 INFO:teuthology.orchestra.run.smithi042.stdout:3699 read 0x525920d thru 0x5259f71 (0xd65 bytes) 2024-09-16T13:00:06.688 INFO:teuthology.orchestra.run.smithi042.stdout:3700 read 0x1a9d880 thru 0x1a9fbed (0x236e bytes) 2024-09-16T13:00:06.689 INFO:teuthology.orchestra.run.smithi042.stdout:3701 writesame 0x1b8f6fd thru 0x1b93880 data_size 0xe(0x4184 bytes) 2024-09-16T13:00:06.695 INFO:teuthology.orchestra.run.smithi042.stdout:3702 punch from 0x4cf9d68 to 0x4d03633, (0x98cb bytes) 2024-09-16T13:00:06.695 INFO:teuthology.orchestra.run.smithi042.stdout:3704 write 0x9e3878d thru 0x9e3a214 (0x1a88 bytes) 2024-09-16T13:00:06.698 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:06.698 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:06.701 INFO:teuthology.orchestra.run.smithi042.stdout:3705 read 0x6c8ac9f thru 0x6c917d6 (0x6b38 bytes) 2024-09-16T13:00:06.701 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.701 INFO:teuthology.orchestra.run.smithi042.stdout:3707 write 0x2b9eb4 thru 0x2bf9d2 (0x5b1f bytes) 2024-09-16T13:00:06.703 INFO:teuthology.orchestra.run.smithi042.stdout:3708 write 0xac20343 thru 0xac22f9b (0x2c59 bytes) 2024-09-16T13:00:06.703 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:06.698+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:06.706 INFO:teuthology.orchestra.run.smithi042.stdout:3709 writesame 0x36b1e59 thru 0x36bab3c data_size 0x7f(0x8ce4 bytes) 2024-09-16T13:00:06.708 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:06.713 INFO:teuthology.orchestra.run.smithi042.stdout:3710 read 0x1a678ac thru 0x1a7332c (0xba81 bytes) 2024-09-16T13:00:06.713 INFO:teuthology.orchestra.run.smithi042.stdout:3711 read 0xa59f382 thru 0xa5ace75 (0xdaf4 bytes) 2024-09-16T13:00:06.714 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.714 INFO:teuthology.orchestra.run.smithi042.stdout:3713 read 0xad9f5fa thru 0xada3d05 (0x470c bytes) 2024-09-16T13:00:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:3715 write 0x3f8671c thru 0x3f8db8d (0x7472 bytes) 2024-09-16T13:00:06.718 INFO:teuthology.orchestra.run.smithi042.stdout:3716 write 0xb38b768 thru 0xb39af70 (0xf809 bytes) 2024-09-16T13:00:06.726 INFO:teuthology.orchestra.run.smithi042.stdout:3717 trunc from 0xc6d97cf to 0xd50777c 2024-09-16T13:00:06.738 INFO:teuthology.orchestra.run.smithi042.stdout:3720 write 0x1506072 thru 0x15069c7 (0x956 bytes) 2024-09-16T13:00:06.744 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.744 INFO:teuthology.orchestra.run.smithi042.stdout:3722 write 0x1fa0173 thru 0x1fa178d (0x161b bytes) 2024-09-16T13:00:06.747 INFO:teuthology.orchestra.run.smithi042.stdout:3723 read 0xa0c898 thru 0xa0f549 (0x2cb2 bytes) 2024-09-16T13:00:06.760 INFO:teuthology.orchestra.run.smithi042.stdout:3725 write 0xdd9472b thru 0xdd97338 (0x2c0e bytes) 2024-09-16T13:00:06.762 INFO:teuthology.orchestra.run.smithi042.stdout:3726 read 0xad93f90 thru 0xad94ce8 (0xd59 bytes) 2024-09-16T13:00:06.763 INFO:teuthology.orchestra.run.smithi042.stdout:3729 punch from 0x4abfd93 to 0x4ac0b29, (0xd96 bytes) 2024-09-16T13:00:06.763 INFO:teuthology.orchestra.run.smithi042.stdout:3730 read 0x6f8889 thru 0x70276d (0x9ee5 bytes) 2024-09-16T13:00:06.775 INFO:teuthology.orchestra.run.smithi042.stdout:3731 write 0xeb226cd thru 0xeb2a15f (0x7a93 bytes) 2024-09-16T13:00:06.779 INFO:teuthology.orchestra.run.smithi042.stdout:3732 punch from 0x672bbbc to 0x672d2c0, (0x1704 bytes) 2024-09-16T13:00:06.793 INFO:teuthology.orchestra.run.smithi042.stdout:3733 trunc from 0xeb2a160 to 0x63a5fdd 2024-09-16T13:00:06.804 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:06.795+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:06.821 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:06.821 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:06.825 INFO:teuthology.orchestra.run.smithi042.stdout:3734 writesame 0xa09e22c thru 0xa0a99e6 data_size 0x17(0xb7bb bytes) 2024-09-16T13:00:06.827 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.832 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:06.838 INFO:teuthology.orchestra.run.smithi042.stdout:3738 trunc from 0xa0a99e7 to 0x2cefd98 2024-09-16T13:00:06.875 INFO:teuthology.orchestra.run.smithi042.stdout:3739 write 0x7da7add thru 0x7daba91 (0x3fb5 bytes) 2024-09-16T13:00:06.877 INFO:teuthology.orchestra.run.smithi042.stdout:3740 write 0x490531e thru 0x4910808 (0xb4eb bytes) 2024-09-16T13:00:06.882 INFO:teuthology.orchestra.run.smithi042.stdout:3741 trunc from 0x7daba92 to 0x8de7975 2024-09-16T13:00:06.893 INFO:teuthology.orchestra.run.smithi042.stdout:3742 trunc from 0x8de7975 to 0x93c6679 2024-09-16T13:00:06.905 INFO:teuthology.orchestra.run.smithi042.stdout:3743 punch from 0x378e92c to 0x3791aa0, (0x3174 bytes) 2024-09-16T13:00:06.905 INFO:teuthology.orchestra.run.smithi042.stdout:3745 write 0x6726fa4 thru 0x67278e5 (0x942 bytes) 2024-09-16T13:00:06.905 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:06.898+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:06.907 INFO:teuthology.orchestra.run.smithi042.stdout:3747 write 0x3118d33 thru 0x311f8cc (0x6b9a bytes) 2024-09-16T13:00:06.913 INFO:teuthology.orchestra.run.smithi042.stdout:3748 punch from 0x2954dc6 to 0x2958cdb, (0x3f15 bytes) 2024-09-16T13:00:06.913 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.913 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.913 INFO:teuthology.orchestra.run.smithi042.stdout:3753 punch from 0x50a4162 to 0x50a5315, (0x11b3 bytes) 2024-09-16T13:00:06.913 INFO:teuthology.orchestra.run.smithi042.stdout:3755 punch from 0x2b67b84 to 0x2b6825a, (0x6d6 bytes) 2024-09-16T13:00:06.913 INFO:teuthology.orchestra.run.smithi042.stdout:3756 writesame 0x1e263c3 thru 0x1e31ea6 data_size 0x36(0xbae4 bytes) 2024-09-16T13:00:06.917 INFO:teuthology.orchestra.run.smithi042.stdout:3758 read 0x5adddaa thru 0x5aed035 (0xf28c bytes) 2024-09-16T13:00:06.917 INFO:teuthology.orchestra.run.smithi042.stdout:3759 read 0x13f118c thru 0x1400064 (0xeed9 bytes) 2024-09-16T13:00:06.930 INFO:teuthology.orchestra.run.smithi042.stdout:3760 trunc from 0x93c6679 to 0x132b839 2024-09-16T13:00:06.950 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:06.950 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:06.953 INFO:teuthology.orchestra.run.smithi042.stdout:3761 punch from 0x743eb7 to 0x75261d, (0xe766 bytes) 2024-09-16T13:00:06.953 INFO:teuthology.orchestra.run.smithi042.stdout:3762 punch from 0x81a284 to 0x8234bf, (0x923b bytes) 2024-09-16T13:00:06.954 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:06.961 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:06.963 INFO:teuthology.orchestra.run.smithi042.stdout:3764 write 0x1e02fda thru 0x1e06113 (0x313a bytes) 2024-09-16T13:00:07.004 INFO:teuthology.orchestra.run.smithi042.stdout:3765 writesame 0xdc86dc3 thru 0xdc8c389 data_size 0x1(0x55c7 bytes) 2024-09-16T13:00:07.004 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.004 INFO:teuthology.orchestra.run.smithi042.stdout:3768 read 0x11fcd5b thru 0x120be7e (0xf124 bytes) 2024-09-16T13:00:07.005 INFO:teuthology.orchestra.run.smithi042.stdout:3770 punch from 0x58b8435 to 0x58c66a9, (0xe274 bytes) 2024-09-16T13:00:07.006 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:06.998+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:07.027 INFO:teuthology.orchestra.run.smithi042.stdout:3771 trunc from 0xdc8c38a to 0x7e449a 2024-09-16T13:00:07.067 INFO:teuthology.orchestra.run.smithi042.stdout:3772 writesame 0x38179ed thru 0x381932c data_size 0x20(0x1940 bytes) 2024-09-16T13:00:07.093 INFO:teuthology.orchestra.run.smithi042.stdout:3774 write 0xaed24bc thru 0xaeda86d (0x83b2 bytes) 2024-09-16T13:00:07.097 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:07.097 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:07.107 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:07.099+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:07.112 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:07.113 INFO:teuthology.orchestra.run.smithi042.stdout:3777 writesame 0xd8a18f8 thru 0xd8abfa4 data_size 0x21(0xa6ad bytes) 2024-09-16T13:00:07.113 INFO:teuthology.orchestra.run.smithi042.stdout:3778 punch from 0x6670b88 to 0x667a3ac, (0x9824 bytes) 2024-09-16T13:00:07.113 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.113 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.117 INFO:teuthology.orchestra.run.smithi042.stdout:3783 trunc from 0xd8abfa5 to 0xae6a178 2024-09-16T13:00:07.132 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.132 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.132 INFO:teuthology.orchestra.run.smithi042.stdout:3788 write 0x3b2ff29 thru 0x3b3c5c6 (0xc69e bytes) 2024-09-16T13:00:07.135 INFO:teuthology.orchestra.run.smithi042.stdout:3789 write 0x5135901 thru 0x5138989 (0x3089 bytes) 2024-09-16T13:00:07.141 INFO:teuthology.orchestra.run.smithi042.stdout:3790 punch from 0x85ad9c9 to 0x85b13b7, (0x39ee bytes) 2024-09-16T13:00:07.141 INFO:teuthology.orchestra.run.smithi042.stdout:3792 read 0x6f9fb59 thru 0x6fa2c2a (0x30d2 bytes) 2024-09-16T13:00:07.141 INFO:teuthology.orchestra.run.smithi042.stdout:3793 write 0x7724ff2 thru 0x7725574 (0x583 bytes) 2024-09-16T13:00:07.144 INFO:teuthology.orchestra.run.smithi042.stdout:3794 read 0x23eba47 thru 0x23f2d56 (0x7310 bytes) 2024-09-16T13:00:07.144 INFO:teuthology.orchestra.run.smithi042.stdout:3795 read 0x2661619 thru 0x2670689 (0xf071 bytes) 2024-09-16T13:00:07.145 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.156 INFO:teuthology.orchestra.run.smithi042.stdout:3797 write 0xbb909ff thru 0xbb91958 (0xf5a bytes) 2024-09-16T13:00:07.179 INFO:teuthology.orchestra.run.smithi042.stdout:3798 trunc from 0xbb91959 to 0x25889b5 2024-09-16T13:00:07.191 INFO:teuthology.orchestra.run.smithi042.stdout:3799 read 0x174734d thru 0x17545ab (0xd25f bytes) 2024-09-16T13:00:07.201 INFO:teuthology.orchestra.run.smithi042.stdout:3800 write 0x28689c7 thru 0x2868ff4 (0x62e bytes) 2024-09-16T13:00:07.207 INFO:teuthology.orchestra.run.smithi042.stdout:3801 read 0x159f90e thru 0x15a9727 (0x9e1a bytes) 2024-09-16T13:00:07.207 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:07.199+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:07.221 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T13:00:07.222 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:07.222 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:07.224 INFO:teuthology.orchestra.run.smithi042.stdout:3803 write 0x56504f2 thru 0x565d4ad (0xcfbc bytes) 2024-09-16T13:00:07.231 INFO:teuthology.orchestra.run.smithi042.stdout:3805 trunc from 0x565d4ae to 0x5f7c972 2024-09-16T13:00:07.233 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:07.261 INFO:teuthology.orchestra.run.smithi042.stdout:3806 writesame 0xbbac02c thru 0xbbbb003 data_size 0x64(0xefd8 bytes) 2024-09-16T13:00:07.264 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.264 INFO:teuthology.orchestra.run.smithi042.stdout:3808 trunc from 0xbbbb004 to 0xbc39692 2024-09-16T13:00:07.275 INFO:teuthology.orchestra.run.smithi042.stdout:3810 read 0xa923090 thru 0xa92e0ba (0xb02b bytes) 2024-09-16T13:00:07.276 INFO:teuthology.orchestra.run.smithi042.stdout:3811 read 0x5b2c8cc thru 0x5b3611a (0x984f bytes) 2024-09-16T13:00:07.276 INFO:teuthology.orchestra.run.smithi042.stdout:3814 writesame 0x665329 thru 0x66df5a data_size 0x25(0x8c32 bytes) 2024-09-16T13:00:07.297 INFO:teuthology.orchestra.run.smithi042.stdout:3817 trunc from 0xbc39692 to 0x2bf38e9 2024-09-16T13:00:07.308 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:07.302+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:07.331 INFO:teuthology.orchestra.run.smithi042.stdout:3818 write 0x8011b10 thru 0x801210d (0x5fe bytes) 2024-09-16T13:00:07.334 INFO:teuthology.orchestra.run.smithi042.stdout:3819 read 0x4675e0a thru 0x4682e17 (0xd00e bytes) 2024-09-16T13:00:07.334 INFO:teuthology.orchestra.run.smithi042.stdout:3820 write 0x77b456b thru 0x77c0789 (0xc21f bytes) 2024-09-16T13:00:07.337 INFO:teuthology.orchestra.run.smithi042.stdout:3821 write 0x36e29cc thru 0x36ea900 (0x7f35 bytes) 2024-09-16T13:00:07.341 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T13:00:07.341 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T13:00:07.341 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-16T13:00:07.341 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T13:00:07.341 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T13:00:07.341 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-16T13:00:07.344 INFO:teuthology.orchestra.run.smithi042.stdout:3822 read 0x6a21db thru 0x6a8f76 (0x6d9c bytes) 2024-09-16T13:00:07.361 INFO:teuthology.orchestra.run.smithi042.stdout:3824 write 0xd1f45b6 thru 0xd20442d (0xfe78 bytes) 2024-09-16T13:00:07.363 INFO:teuthology.orchestra.run.smithi042.stdout:3826 write 0xa5c8c31 thru 0xa5cdf96 (0x5366 bytes) 2024-09-16T13:00:07.366 INFO:teuthology.orchestra.run.smithi042.stdout:3828 writesame 0xb98b768 thru 0xb99ac8d data_size 0x1(0xf526 bytes) 2024-09-16T13:00:07.382 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:07.382 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:07.391 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:00:07.426 INFO:teuthology.orchestra.run.smithi042.stdout:3831 trunc from 0xd20442e to 0x65aa48f 2024-09-16T13:00:07.427 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:07.419+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:07.446 INFO:teuthology.orchestra.run.smithi042.stdout:3832 trunc from 0x65aa48f to 0xab53b2c 2024-09-16T13:00:07.452 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:07.452 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T13:00:07.452 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:07.456 INFO:teuthology.orchestra.run.smithi042.stdout:3833 read 0x2a87cb8 thru 0x2a92036 (0xa37f bytes) 2024-09-16T13:00:07.457 INFO:teuthology.orchestra.run.smithi042.stdout:3834 write 0x27004d7 thru 0x2706790 (0x62ba bytes) 2024-09-16T13:00:07.463 INFO:teuthology.orchestra.run.smithi042.stdout:3835 punch from 0x50efea9 to 0x50fae3d, (0xaf94 bytes) 2024-09-16T13:00:07.477 INFO:teuthology.orchestra.run.smithi042.stdout:3838 trunc from 0xab53b2c to 0x21ff729 2024-09-16T13:00:07.499 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:07.499 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:07.508 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:00:07.509 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:07.499+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:07.514 INFO:teuthology.orchestra.run.smithi042.stdout:3839 writesame 0x5b2e297 thru 0x5b34954 data_size 0x1(0x66be bytes) 2024-09-16T13:00:07.514 INFO:teuthology.orchestra.run.smithi042.stdout:3840 punch from 0x288b3b0 to 0x2895383, (0x9fd3 bytes) 2024-09-16T13:00:07.514 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.514 INFO:teuthology.orchestra.run.smithi042.stdout:3842 punch from 0x4721ceb to 0x472ac7c, (0x8f91 bytes) 2024-09-16T13:00:07.529 INFO:teuthology.orchestra.run.smithi042.stdout:3843 trunc from 0x5b34955 to 0xea38834 2024-09-16T13:00:07.539 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.539 INFO:teuthology.orchestra.run.smithi042.stdout:3845 write 0x2634ee5 thru 0x26359fc (0xb18 bytes) 2024-09-16T13:00:07.544 INFO:teuthology.orchestra.run.smithi042.stdout:3846 write 0x6ea3e53 thru 0x6eb011c (0xc2ca bytes) 2024-09-16T13:00:07.549 INFO:teuthology.orchestra.run.smithi042.stdout:3847 write 0x3ab8a20 thru 0x3abe978 (0x5f59 bytes) 2024-09-16T13:00:07.551 INFO:teuthology.orchestra.run.smithi042.stdout:3850 punch from 0x93c201f to 0x93cfa99, (0xda7a bytes) 2024-09-16T13:00:07.552 INFO:teuthology.orchestra.run.smithi042.stdout:3854 read 0x65664f9 thru 0x656d582 (0x708a bytes) 2024-09-16T13:00:07.552 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.552 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.552 INFO:teuthology.orchestra.run.smithi042.stdout:3857 write 0xcd0308f thru 0xcd0384e (0x7c0 bytes) 2024-09-16T13:00:07.558 INFO:teuthology.orchestra.run.smithi042.stdout:3859 trunc from 0xea38834 to 0xc363c2b 2024-09-16T13:00:07.584 INFO:teuthology.orchestra.run.smithi042.stdout:3861 writesame 0xdf899dc thru 0xdf8a166 data_size 0x1(0x78b bytes) 2024-09-16T13:00:07.586 INFO:teuthology.orchestra.run.smithi042.stdout:3862 writesame 0xa76e8ee thru 0xa7741bf data_size 0x1(0x58d2 bytes) 2024-09-16T13:00:07.600 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:07.601 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:07.609 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:00:07.610 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:07.606+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:07.615 INFO:teuthology.orchestra.run.smithi042.stdout:3864 writesame 0xebd6e11 thru 0xebdd73e data_size 0x1(0x692e bytes) 2024-09-16T13:00:07.615 INFO:teuthology.orchestra.run.smithi042.stdout:3865 punch from 0x771281b to 0x771f15e, (0xc943 bytes) 2024-09-16T13:00:07.615 INFO:teuthology.orchestra.run.smithi042.stdout:3866 write 0xdd733c0 thru 0xdd76e52 (0x3a93 bytes) 2024-09-16T13:00:07.617 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.618 INFO:teuthology.orchestra.run.smithi042.stdout:3870 read 0xe5cf762 thru 0xe5d3b55 (0x43f4 bytes) 2024-09-16T13:00:07.618 INFO:teuthology.orchestra.run.smithi042.stdout:3875 write 0xbed59df thru 0xbee3c16 (0xe238 bytes) 2024-09-16T13:00:07.625 INFO:teuthology.orchestra.run.smithi042.stdout:3877 writesame 0x411ff40 thru 0x41214e4 data_size 0x1(0x15a5 bytes) 2024-09-16T13:00:07.637 INFO:teuthology.orchestra.run.smithi042.stdout:3878 read 0xdd983c thru 0xde73a9 (0xdb6e bytes) 2024-09-16T13:00:07.654 INFO:teuthology.orchestra.run.smithi042.stdout:3879 trunc from 0xebdd73f to 0x5cdee23 2024-09-16T13:00:07.688 INFO:teuthology.orchestra.run.smithi042.stdout:3880 writesame 0xa749373 thru 0xa751d6c data_size 0x57(0x89fa bytes) 2024-09-16T13:00:07.690 INFO:teuthology.orchestra.run.smithi042.stdout:3881 writesame 0x9ec9c35 thru 0x9ed72b7 data_size 0x23(0xd683 bytes) 2024-09-16T13:00:07.696 INFO:teuthology.orchestra.run.smithi042.stdout:3882 read 0x4ececfc thru 0x4ed6e0a (0x810f bytes) 2024-09-16T13:00:07.697 INFO:teuthology.orchestra.run.smithi042.stdout:3883 write 0x5db523e thru 0x5dc132e (0xc0f1 bytes) 2024-09-16T13:00:07.700 INFO:teuthology.orchestra.run.smithi042.stdout:3885 read 0xa51d59a thru 0xa527fac (0xaa13 bytes) 2024-09-16T13:00:07.700 INFO:teuthology.orchestra.run.smithi042.stdout:3886 read 0xa2922f0 thru 0xa29dfde (0xbcef bytes) 2024-09-16T13:00:07.701 INFO:teuthology.orchestra.run.smithi042.stdout:3887 write 0x4ff223c thru 0x500040d (0xe1d2 bytes) 2024-09-16T13:00:07.703 INFO:teuthology.orchestra.run.smithi042.stdout:3888 write 0x223b456 thru 0x223f77b (0x4326 bytes) 2024-09-16T13:00:07.707 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:07.707 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:07.710 INFO:teuthology.orchestra.run.smithi042.stdout:3889 write 0x5a6850e thru 0x5a68585 (0x78 bytes) 2024-09-16T13:00:07.711 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:07.703+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:07.713 INFO:teuthology.orchestra.run.smithi042.stdout:3890 punch from 0x7b0deed to 0x7b13f79, (0x608c bytes) 2024-09-16T13:00:07.713 INFO:teuthology.orchestra.run.smithi042.stdout:3892 write 0x61a183e thru 0x61abf1f (0xa6e2 bytes) 2024-09-16T13:00:07.718 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:07.719 INFO:teuthology.orchestra.run.smithi042.stdout:3894 write 0x78095e1 thru 0x7818246 (0xec66 bytes) 2024-09-16T13:00:07.723 INFO:teuthology.orchestra.run.smithi042.stdout:3895 write 0x32fc4e2 thru 0x3306e97 (0xa9b6 bytes) 2024-09-16T13:00:07.749 INFO:teuthology.orchestra.run.smithi042.stdout:3896 write 0xc953315 thru 0xc95ff71 (0xcc5d bytes) 2024-09-16T13:00:07.753 INFO:teuthology.orchestra.run.smithi042.stdout:3897 punch from 0x20c904b to 0x20ced62, (0x5d17 bytes) 2024-09-16T13:00:07.753 INFO:teuthology.orchestra.run.smithi042.stdout:3898 punch from 0x7ff1d2f to 0x7ffe89b, (0xcb6c bytes) 2024-09-16T13:00:07.754 INFO:teuthology.orchestra.run.smithi042.stdout:3899 trunc from 0xc95ff72 to 0xd40805a 2024-09-16T13:00:07.768 INFO:teuthology.orchestra.run.smithi042.stdout:3900 read 0x805effe thru 0x806bf91 (0xcf94 bytes) 2024-09-16T13:00:07.769 INFO:teuthology.orchestra.run.smithi042.stdout:3901 read 0x86e1a46 thru 0x86f0f94 (0xf54f bytes) 2024-09-16T13:00:07.769 INFO:teuthology.orchestra.run.smithi042.stdout:3902 punch from 0x628bd56 to 0x628d556, (0x1800 bytes) 2024-09-16T13:00:07.770 INFO:teuthology.orchestra.run.smithi042.stdout:3903 writesame 0x24dd1ae thru 0x24e2260 data_size 0x1(0x50b3 bytes) 2024-09-16T13:00:07.785 INFO:teuthology.orchestra.run.smithi042.stdout:3905 read 0xb2105a5 thru 0xb217d99 (0x77f5 bytes) 2024-09-16T13:00:07.786 INFO:teuthology.orchestra.run.smithi042.stdout:3907 read 0xc05cf44 thru 0xc06077e (0x383b bytes) 2024-09-16T13:00:07.787 INFO:teuthology.orchestra.run.smithi042.stdout:3908 read 0x8baceb1 thru 0x8baf130 (0x2280 bytes) 2024-09-16T13:00:07.787 INFO:teuthology.orchestra.run.smithi042.stdout:3909 read 0x4278ac5 thru 0x427d40e (0x494a bytes) 2024-09-16T13:00:07.797 INFO:teuthology.orchestra.run.smithi042.stdout:3910 trunc from 0xd40805a to 0x8e0e4b1 2024-09-16T13:00:07.812 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:07.806+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:07.812 INFO:teuthology.orchestra.run.smithi042.stdout:3912 writesame 0x1b82042 thru 0x1b867a0 data_size 0xb(0x475f bytes) 2024-09-16T13:00:07.814 INFO:teuthology.orchestra.run.smithi042.stdout:3913 write 0x3231562 thru 0x3232f11 (0x19b0 bytes) 2024-09-16T13:00:07.815 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:07.815 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:07.816 INFO:teuthology.orchestra.run.smithi042.stdout:3914 read 0xffd119 thru 0x100cb4f (0xfa37 bytes) 2024-09-16T13:00:07.817 INFO:teuthology.orchestra.run.smithi042.stdout:3915 write 0x130a34e thru 0x13181fe (0xdeb1 bytes) 2024-09-16T13:00:07.819 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:07.825 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:07.841 INFO:teuthology.orchestra.run.smithi042.stdout:3920 writesame 0xe74a18e thru 0xe74eb4a data_size 0x1(0x49bd bytes) 2024-09-16T13:00:07.843 INFO:teuthology.orchestra.run.smithi042.stdout:3921 read 0x5a4df24 thru 0x5a594a6 (0xb583 bytes) 2024-09-16T13:00:07.844 INFO:teuthology.orchestra.run.smithi042.stdout:3922 writesame 0x9bd7156 thru 0x9bdf961 data_size 0x2(0x880c bytes) 2024-09-16T13:00:07.860 INFO:teuthology.orchestra.run.smithi042.stdout:3923 read 0x8a824fa thru 0x8a8d4fd (0xb004 bytes) 2024-09-16T13:00:07.875 INFO:teuthology.orchestra.run.smithi042.stdout:3924 trunc from 0xe74eb4b to 0x68c5bb1 2024-09-16T13:00:07.912 INFO:teuthology.orchestra.run.smithi042.stdout:3925 write 0xcad5143 thru 0xcadbed4 (0x6d92 bytes) 2024-09-16T13:00:07.912 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:07.906+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:07.916 INFO:teuthology.orchestra.run.smithi042.stdout:3927 read 0xb7ee85d thru 0xb7f408e (0x5832 bytes) 2024-09-16T13:00:07.916 INFO:teuthology.orchestra.run.smithi042.stdout:3928 read 0x8042251 thru 0x8047c01 (0x59b1 bytes) 2024-09-16T13:00:07.924 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:07.924 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:07.933 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:07.938 INFO:teuthology.orchestra.run.smithi042.stdout:3929 trunc from 0xcadbed5 to 0x22402d 2024-09-16T13:00:07.990 INFO:teuthology.orchestra.run.smithi042.stdout:3930 trunc from 0x22402d to 0x139f581 2024-09-16T13:00:08.002 INFO:teuthology.orchestra.run.smithi042.stdout:3932 punch from 0x46706a to 0x4759ca, (0xe960 bytes) 2024-09-16T13:00:08.002 INFO:teuthology.orchestra.run.smithi042.stdout:3933 trunc from 0x139f581 to 0x152c88a 2024-09-16T13:00:08.013 INFO:teuthology.orchestra.run.smithi042.stdout:3935 punch from 0x705e06 to 0x71076c, (0xa966 bytes) 2024-09-16T13:00:08.013 INFO:teuthology.orchestra.run.smithi042.stdout:3938 punch from 0xb21ce5 to 0xb230f1, (0x140c bytes) 2024-09-16T13:00:08.013 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:08.006+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:08.019 INFO:teuthology.orchestra.run.smithi042.stdout:3939 trunc from 0x152c88a to 0x4493e59 2024-09-16T13:00:08.032 INFO:teuthology.orchestra.run.smithi042.stdout:3942 read 0xa1db54 thru 0xa1dc32 (0xdf bytes) 2024-09-16T13:00:08.047 INFO:teuthology.orchestra.run.smithi042.stdout:3943 writesame 0x4c3091d thru 0x4c37af6 data_size 0x3b(0x71da bytes) 2024-09-16T13:00:08.047 INFO:teuthology.orchestra.run.smithi042.stdout:3944 read 0x498c435 thru 0x49958bc (0x9488 bytes) 2024-09-16T13:00:08.047 INFO:teuthology.orchestra.run.smithi042.stdout:3945 read 0x45270c6 thru 0x4534783 (0xd6be bytes) 2024-09-16T13:00:08.047 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:08.047 INFO:teuthology.orchestra.run.smithi042.stdout:3948 read 0x39060b2 thru 0x3910ccf (0xac1e bytes) 2024-09-16T13:00:08.048 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:08.048 INFO:teuthology.orchestra.run.smithi042.stdout:3953 write 0x4c0e2e9 thru 0x4c1e04b (0xfd63 bytes) 2024-09-16T13:00:08.077 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:08.078 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:08.078 INFO:teuthology.orchestra.run.smithi042.stdout:3954 writesame 0x7aa1178 thru 0x7aafad0 data_size 0x29(0xe959 bytes) 2024-09-16T13:00:08.079 INFO:teuthology.orchestra.run.smithi042.stdout:3955 read 0x3fe54b4 thru 0x3febd81 (0x68ce bytes) 2024-09-16T13:00:08.079 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:08.079 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:08.079 INFO:teuthology.orchestra.run.smithi042.stdout:3958 clone 21 order 20 su 8192 sc 2 2024-09-16T13:00:08.085 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:08.114 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:08.107+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:08.184 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T13:00:08.187 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:08.187 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:08.197 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:08.214 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:08.210+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:08.291 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:08.292 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:08.301 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:08.346 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:08.342+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:08.400 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:08.400 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:08.409 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:08.411 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-16T13:00:08.416 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:08.407+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:08.502 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:08.502 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:08.511 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:08.516 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:08.507+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:08.618 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:08.610+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:08.618 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:08.618 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:08.627 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:08.634 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:08.718 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:08.714+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:08.775 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:08.775 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:08.775 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:08.784 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:08.800 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:08.819 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:08.814+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:08.820 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:08.838 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:08.859 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:00:08.868 INFO:teuthology.orchestra.run.smithi042.stdout:3959 write 0x26651b6 thru 0x266dc75 (0x8ac0 bytes) 2024-09-16T13:00:08.885 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:08.887 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:08.888 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:08.899 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:08.900 INFO:teuthology.orchestra.run.smithi042.stdout:3961 write 0x8e09ff8 thru 0x8e192c2 (0xf2cb bytes) 2024-09-16T13:00:08.910 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:08.918 INFO:teuthology.orchestra.run.smithi042.stdout:3962 write 0x92aa985 thru 0x92ad1b8 (0x2834 bytes) 2024-09-16T13:00:08.919 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:08.911+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:08.928 INFO:teuthology.orchestra.run.smithi042.stdout:3963 read 0x568eeeb thru 0x5692e89 (0x3f9f bytes) 2024-09-16T13:00:08.928 INFO:teuthology.orchestra.run.smithi042.stdout:3964 writesame 0x628d301 thru 0x629cb90 data_size 0xa4(0xf890 bytes) 2024-09-16T13:00:08.932 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:08.935 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:08.935 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:08.935 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:08.935 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:08.935 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:00:08.935 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:08.936 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:08.936 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:08.939 INFO:teuthology.orchestra.run.smithi042.stdout:3967 read 0x79baf51 thru 0x79c3786 (0x8836 bytes) 2024-09-16T13:00:08.939 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:08.939 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:08.950 INFO:teuthology.orchestra.run.smithi042.stdout:3970 trunc from 0x92ad1b9 to 0x2fe8127 2024-09-16T13:00:08.981 INFO:teuthology.orchestra.run.smithi042.stdout:3971 read 0xc608bf thru 0xc6b41d (0xab5f bytes) 2024-09-16T13:00:09.002 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:09.002 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:09.003 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:09.003 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:09.003 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:09.007 INFO:teuthology.orchestra.run.smithi042.stdout:3972 write 0xc8b86cf thru 0xc8bc1d7 (0x3b09 bytes) 2024-09-16T13:00:09.013 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:09.017 INFO:teuthology.orchestra.run.smithi042.stdout:3973 read 0x347f58d thru 0x348c133 (0xcba7 bytes) 2024-09-16T13:00:09.017 INFO:teuthology.orchestra.run.smithi042.stdout:3974 punch from 0x46b936b to 0x46ba946, (0x15db bytes) 2024-09-16T13:00:09.017 INFO:teuthology.orchestra.run.smithi042.stdout:3975 write 0x7e5f3d0 thru 0x7e6b55a (0xc18b bytes) 2024-09-16T13:00:09.019 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:09.011+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:09.033 INFO:teuthology.orchestra.run.smithi042.stdout:3980 trunc from 0xc8bc1d8 to 0x78fcadf 2024-09-16T13:00:09.054 INFO:teuthology.orchestra.run.smithi042.stdout:3981 punch from 0x33af27b to 0x33b76d0, (0x8455 bytes) 2024-09-16T13:00:09.068 INFO:teuthology.orchestra.run.smithi042.stdout:3982 trunc from 0x78fcadf to 0x4ac014 2024-09-16T13:00:09.103 INFO:teuthology.orchestra.run.smithi042.stdout:3985 trunc from 0x4ac014 to 0x3eaad30 2024-09-16T13:00:09.113 INFO:teuthology.orchestra.run.smithi042.stdout:3986 writesame 0x17dbdff thru 0x17dc7b4 data_size 0x16(0x9b6 bytes) 2024-09-16T13:00:09.120 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:09.114+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:09.123 INFO:teuthology.orchestra.run.smithi042.stdout:3987 read 0x33a58d0 thru 0x33af269 (0x999a bytes) 2024-09-16T13:00:09.126 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:09.126 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:09.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:09.126 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:09.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.126 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:636 4.3 4:c7f511a6:::rbd_data.11b59d02cc38.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T13:00:09.126 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:17.472199+0000", 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.649148132000001, 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027791489999999999, 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 636 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T13:00:09.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472199+0000", 2024-09-16T13:00:09.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472202+0000", 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.01e-06 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472218+0000", 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5724000000000001e-05 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472261+0000", 2024-09-16T13:00:09.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2858999999999997e-05 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472277+0000", 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6297999999999999e-05 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472397+0000", 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012026199999999999 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.472563+0000", 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016614900000000001 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.480681+0000", 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081177869999999996 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499671+0000", 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018990289 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499756+0000", 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4511000000000005e-05 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499858+0000", 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102495 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:17.499991+0000", 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132106 2024-09-16T13:00:09.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:3577 4.1e 4:7a8a5a14:::rbd_data.11b5c378985b.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:22.100645+0000", 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.020701680000002, 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027798529999999998, 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3577 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100645+0000", 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100649+0000", 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1769999999999998e-06 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100659+0000", 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0407e-05 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100689+0000", 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0290999999999998e-05 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100701+0000", 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1960999999999999e-05 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.100800+0000", 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8350000000000005e-05 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.101027+0000", 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000227736 2024-09-16T13:00:09.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.111830+0000", 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010802735000000001 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128285+0000", 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016454389999999999 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128338+0000", 2024-09-16T13:00:09.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3112000000000002e-05 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128398+0000", 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0136999999999997e-05 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:22.128444+0000", 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6233999999999998e-05 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5229 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:27.838222+0000", 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.283125353000003, 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049016400000000002, 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5229 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838222+0000", 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838225+0000", 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0340000000000001e-06 2024-09-16T13:00:09.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838239+0000", 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4004e-05 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838280+0000", 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0738000000000002e-05 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838294+0000", 2024-09-16T13:00:09.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.482e-05 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838415+0000", 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012084 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.838570+0000", 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000155162 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:09.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.850428+0000", 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011857826 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887135+0000", 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036706938000000001 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887149+0000", 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3421000000000001e-05 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887202+0000", 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3427000000000002e-05 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:27.887238+0000", 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6189999999999997e-05 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:5396 4.1e 4:7ab6716d:::rbd_data.11b53fd98b75.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:09.145 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:28.222043+0000", 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.899304229999998, 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035598629999999999, 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5396 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T13:00:09.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222043+0000", 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222045+0000", 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9580000000000001e-06 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222055+0000", 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9310000000000003e-06 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222082+0000", 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6882000000000001e-05 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222092+0000", 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8910000000000007e-06 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222166+0000", 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4437999999999994e-05 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.222264+0000", 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7748e-05 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.228318+0000", 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060540109999999998 2024-09-16T13:00:09.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257411+0000", 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029093622999999999 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257500+0000", 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8341e-05 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257580+0000", 2024-09-16T13:00:09.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:28.257642+0000", 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1073000000000003e-05 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:18283 4.1e 4:7c33f2a6:::rbd_data.11b51a3eb68b.000000000000002b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:56.586760+0000", 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.534586702, 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.031869502000000001, 2024-09-16T13:00:09.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18283 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586760+0000", 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586760+0000", 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586763+0000", 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0170000000000001e-06 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586777+0000", 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3692999999999999e-05 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586817+0000", 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9796999999999999e-05 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586856+0000", 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8829999999999999e-05 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.586975+0000", 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119083 2024-09-16T13:00:09.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.587231+0000", 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025600999999999998 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.596243+0000", 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090123749999999996 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.618539+0000", 2024-09-16T13:00:09.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022296137000000001 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.618548+0000", 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9770000000000006e-06 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.618588+0000", 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9681999999999997e-05 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:56.618630+0000", 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1900999999999997e-05 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:27166 4.6 4:67a7dc0b:::rbd_data.11b59afb69bb.0000000000000006:16 [read 2195456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.567014+0000", 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.5543332260000007, 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020993722999999999, 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 27166 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567014+0000", 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567014+0000", 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567014+0000", 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567017+0000", 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3819999999999998e-06 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567024+0000", 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9480000000000002e-06 2024-09-16T13:00:09.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.567028+0000", 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9700000000000001e-06 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.582236+0000", 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015208172000000001 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.582312+0000", 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5903999999999997e-05 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.588008+0000", 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056959089999999999 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32029 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3543040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.736668+0000", 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3846796519999991, 2024-09-16T13:00:09.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022092409, 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32029 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736668+0000", 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736668+0000", 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736668+0000", 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3e-07 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736671+0000", 2024-09-16T13:00:09.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5069999999999999e-06 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736678+0000", 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4200000000000001e-06 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736682+0000", 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8840000000000004e-06 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.755361+0000", 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018678555999999999 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.755484+0000", 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306599999999999 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.758760+0000", 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003276246 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32033 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.736733+0000", 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3846146140000002, 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019686207000000001, 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32033 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.166 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736733+0000", 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736733+0000", 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736733+0000", 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6100000000000002e-07 2024-09-16T13:00:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736735+0000", 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.973e-06 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736741+0000", 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.736746+0000", 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2490000000000001e-06 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756236+0000", 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019490176000000001 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756337+0000", 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100455 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756419+0000", 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2103999999999999e-05 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32063 4.6 4:67a7dc0b:::rbd_data.11b59afb69bb.0000000000000006:16 [read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737067+0000", 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3842798829999996, 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019817906999999999, 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32063 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737067+0000", 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737067+0000", 2024-09-16T13:00:09.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737068+0000", 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737070+0000", 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2649999999999999e-06 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737076+0000", 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7610000000000004e-06 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737079+0000", 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6650000000000001e-06 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756739+0000", 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019659421999999999 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756822+0000", 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3013e-05 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756885+0000", 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3282999999999997e-05 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32064 4.6 4:67a7dc0b:::rbd_data.11b59afb69bb.0000000000000006:16 [read 3633152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.174 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737083+0000", 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3842637230000001, 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019903495, 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32064 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737083+0000", 2024-09-16T13:00:09.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737083+0000", 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737084+0000", 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737086+0000", 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2409999999999998e-06 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737092+0000", 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8109999999999999e-06 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737095+0000", 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5379999999999998e-06 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756913+0000", 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019817952999999999 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756954+0000", 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0676000000000002e-05 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.756987+0000", 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2898000000000002e-05 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.178 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32065 4.6 4:67a7dc0b:::rbd_data.11b59afb69bb.0000000000000006:16 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737099+0000", 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3842477249999998, 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020025591999999998, 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32065 2024-09-16T13:00:09.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737099+0000", 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737099+0000", 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737100+0000", 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737102+0000", 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0619999999999999e-06 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737108+0000", 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6679999999999998e-06 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737111+0000", 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4010000000000001e-06 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757000+0000", 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019888922999999999 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757058+0000", 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7595e-05 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757125+0000", 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7442999999999998e-05 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32093 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3547136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737564+0000", 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3837831230000006, 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019709436, 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.183 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32093 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737564+0000", 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737564+0000", 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737565+0000", 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.47e-07 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737570+0000", 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8389999999999997e-06 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737583+0000", 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2843000000000001e-05 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737589+0000", 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0800000000000002e-06 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757146+0000", 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019557261999999999 2024-09-16T13:00:09.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757192+0000", 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5754000000000003e-05 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757273+0000", 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1811000000000007e-05 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.187 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32094 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737595+0000", 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3837522230000001, 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019827239999999999, 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32094 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737595+0000", 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737595+0000", 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737595+0000", 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737598+0000", 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8159999999999998e-06 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737605+0000", 2024-09-16T13:00:09.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3810000000000004e-06 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737609+0000", 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0890000000000002e-06 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757295+0000", 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019686454999999999 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757346+0000", 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0952999999999998e-05 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757422+0000", 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6015999999999999e-05 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32095 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3604480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.192 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737613+0000", 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3837342150000005, 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019949204000000002, 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32095 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737613+0000", 2024-09-16T13:00:09.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737613+0000", 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737614+0000", 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7899999999999998e-07 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737616+0000", 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9080000000000002e-06 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737622+0000", 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1920000000000002e-06 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737626+0000", 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0199999999999996e-06 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757453+0000", 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019827506000000002 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757515+0000", 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1254999999999995e-05 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757562+0000", 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7644e-05 2024-09-16T13:00:09.196 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32096 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737630+0000", 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3837171309999992, 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020104349000000001, 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.197 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32096 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737630+0000", 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737630+0000", 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737630+0000", 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737632+0000", 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0490000000000002e-06 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737638+0000", 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7810000000000002e-06 2024-09-16T13:00:09.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737642+0000", 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.692e-06 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757580+0000", 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019937723000000001 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757656+0000", 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.606e-05 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757734+0000", 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8658999999999993e-05 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:32097 4.6 4:61a6b840:::rbd_data.11b59afb69bb.000000000000000d:16 [read 3661824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:59.737646+0000", 2024-09-16T13:00:09.201 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3837012360000003, 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020224474999999999, 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 32097 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737646+0000", 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737646+0000", 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737646+0000", 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8999999999999997e-07 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737648+0000", 2024-09-16T13:00:09.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.979e-06 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737655+0000", 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6959999999999999e-06 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.737659+0000", 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9670000000000003e-06 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757757+0000", 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020098443000000001 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757819+0000", 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1369000000000001e-05 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:59.757870+0000", 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1530999999999998e-05 2024-09-16T13:00:09.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42592 4.6 4:611ecd9e:::rbd_data.11b52dde2e.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.300454+0000", 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.82089341, 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025747084999999999, 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.206 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42592 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300454+0000", 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300454+0000", 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300455+0000", 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4720000000000001e-06 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300462+0000", 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0520000000000004e-06 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300483+0000", 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0466e-05 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300491+0000", 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5380000000000002e-06 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300553+0000", 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2150999999999995e-05 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.300655+0000", 2024-09-16T13:00:09.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010206599999999999 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.304922+0000", 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042666730000000003 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.326046+0000", 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021124079 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.326072+0000", 2024-09-16T13:00:09.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6213000000000001e-05 2024-09-16T13:00:09.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.211 INFO:teuthology.orchestra.run.smithi042.stdout:3988 trunc from 0x3eaad30 to 0x54191b3 2024-09-16T13:00:09.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.326146+0000", 2024-09-16T13:00:09.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3267000000000002e-05 2024-09-16T13:00:09.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.326201+0000", 2024-09-16T13:00:09.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5108000000000002e-05 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42607 4.3 4:c3aab101:::rbd_data.11b52dde2e.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.345306+0000", 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.7760415819999995, 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036778706000000001, 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42607 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345306+0000", 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345306+0000", 2024-09-16T13:00:09.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345307+0000", 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4300000000000001e-06 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345314+0000", 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3000000000000004e-06 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345334+0000", 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9794000000000001e-05 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345342+0000", 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9519999999999994e-06 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345398+0000", 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6347000000000001e-05 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.345471+0000", 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2399999999999998e-05 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.352464+0000", 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069934630000000001 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.381945+0000", 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029481120999999999 2024-09-16T13:00:09.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.381962+0000", 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7030000000000001e-05 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.382034+0000", 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1693999999999995e-05 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.382084+0000", 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0175e-05 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44624 4.6 4:611ecd9e:::rbd_data.11b52dde2e.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:05.550726+0000", 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5706211899999998, 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041671303999999999, 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44624 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550726+0000", 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550726+0000", 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550729+0000", 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.49e-06 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550746+0000", 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.632e-05 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550788+0000", 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2549999999999997e-05 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550804+0000", 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6140000000000001e-05 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550895+0000", 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0585000000000001e-05 2024-09-16T13:00:09.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.550982+0000", 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6916999999999995e-05 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.558031+0000", 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070490539999999999 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.592241+0000", 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.034209943999999999 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.592296+0000", 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4882000000000001e-05 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.592354+0000", 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8295999999999999e-05 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.592397+0000", 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3126e-05 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44860 4.6 4:611ecd9e:::rbd_data.11b52dde2e.0000000000000003:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:06.021827+0000", 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.0995202989999999, 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030066649000000001, 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44860 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021827+0000", 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021827+0000", 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021829+0000", 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.543e-06 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021839+0000", 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7370000000000001e-06 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021864+0000", 2024-09-16T13:00:09.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4850000000000001e-05 2024-09-16T13:00:09.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021878+0000", 2024-09-16T13:00:09.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3935e-05 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.021962+0000", 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4201000000000002e-05 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.022043+0000", 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1241000000000004e-05 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.027176+0000", 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051329009999999996 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.051817+0000", 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024640815999999999 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.051825+0000", 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1580000000000005e-06 2024-09-16T13:00:09.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.051867+0000", 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1499999999999999e-05 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.051894+0000", 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6767e-05 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.232 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.232 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:09.232 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:09.233 INFO:teuthology.orchestra.run.smithi042.stdout:3989 read 0x48c420d thru 0x48d0a47 (0xc83b bytes) 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:3990 write 0x35769fe thru 0x3585700 (0xed03 bytes) 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:3992 write 0x87c776a thru 0x87c7e91 (0x728 bytes) 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:3995 write 0xdf162ef thru 0xdf1d064 (0x6d76 bytes) 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:3996 write 0xce1dd17 thru 0xce1fa7f (0x1d69 bytes) 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:3999 punch from 0x35b162 to 0x36118e, (0x602c bytes) 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:4003 write 0x4c6429 thru 0x4c6d0f (0x8e7 bytes) 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:4004 write 0xbe06474 thru 0xbe149b9 (0xe546 bytes) 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:4006 write 0x5912e02 thru 0x591c719 (0x9918 bytes) 2024-09-16T13:00:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:4009 write 0x778ba3b thru 0x778cae7 (0x10ad bytes) 2024-09-16T13:00:09.236 INFO:teuthology.orchestra.run.smithi042.stdout:4012 write 0x34d311a thru 0x34d963c (0x6523 bytes) 2024-09-16T13:00:09.236 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:09.227+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:09.240 INFO:teuthology.orchestra.run.smithi042.stdout:4013 write 0x1270f41 thru 0x1279ddf (0x8e9f bytes) 2024-09-16T13:00:09.250 INFO:teuthology.orchestra.run.smithi042.stdout:4014 clone 22 order 22 su 16384 sc 3 2024-09-16T13:00:09.337 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:09.327+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:09.340 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:09.340 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:09.341 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:09.341 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:09.350 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:09.438 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:09.431+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:09.501 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:09.501 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:09.501 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:09.501 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:09.501 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:09.510 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:09.539 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:09.534+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:09.613 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17072 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309263+0000", 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.298272825000002, 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018917891999999999, 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17072 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309263+0000", 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309263+0000", 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309263+0000", 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309264+0000", 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.065e-06 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309267+0000", 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8100000000000002e-06 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309269+0000", 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8819999999999999e-06 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.327973+0000", 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018703930000000001 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328065+0000", 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2188000000000006e-05 2024-09-16T13:00:09.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328181+0000", 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115648 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17073 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309271+0000", 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.298264869, 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019262013000000001, 2024-09-16T13:00:09.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17073 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309271+0000", 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309271+0000", 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309271+0000", 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5400000000000002e-07 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309272+0000", 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.540000000000001e-07 2024-09-16T13:00:09.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309275+0000", 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309277+0000", 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.95e-06 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328225+0000", 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018948901000000001 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328318+0000", 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2245999999999997e-05 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328533+0000", 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021494099999999999 2024-09-16T13:00:09.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17074 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309278+0000", 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.298257139, 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019430942, 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17074 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309278+0000", 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309278+0000", 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309279+0000", 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4900000000000002e-07 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309280+0000", 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.426e-06 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309283+0000", 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8619999999999999e-06 2024-09-16T13:00:09.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309285+0000", 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328540+0000", 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019254789000000001 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328641+0000", 2024-09-16T13:00:09.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010133599999999999 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328709+0000", 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8353000000000001e-05 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17075 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309287+0000", 2024-09-16T13:00:09.627 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.298248888, 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019911412, 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17075 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309287+0000", 2024-09-16T13:00:09.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309287+0000", 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309287+0000", 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6600000000000003e-07 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309288+0000", 2024-09-16T13:00:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0449999999999999e-06 2024-09-16T13:00:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309291+0000", 2024-09-16T13:00:09.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8459999999999999e-06 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309293+0000", 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9719999999999999e-06 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328763+0000", 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019470357000000001 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.328982+0000", 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021875700000000001 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329198+0000", 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216169 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.632 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17076 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309295+0000", 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.298240533000001, 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020043858000000001, 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17076 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309295+0000", 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309295+0000", 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309295+0000", 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5199999999999998e-07 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309296+0000", 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0380000000000001e-06 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309299+0000", 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.926e-06 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309301+0000", 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8929999999999999e-06 2024-09-16T13:00:09.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329222+0000", 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019920617000000002 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329276+0000", 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4743000000000001e-05 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329339+0000", 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2389000000000002e-05 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17077 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309303+0000", 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.298232442, 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020302292999999999, 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17077 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309303+0000", 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309303+0000", 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309303+0000", 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5600000000000002e-07 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309304+0000", 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6899999999999996e-07 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309307+0000", 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0089999999999999e-06 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309309+0000", 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0990000000000001e-06 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329377+0000", 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020068004 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329477+0000", 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100134 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329605+0000", 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012782200000000001 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17078 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309311+0000", 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.298224173000001, 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020446052999999999, 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17078 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309311+0000", 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309311+0000", 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309312+0000", 2024-09-16T13:00:09.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4200000000000002e-07 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309313+0000", 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105e-06 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309316+0000", 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8679999999999999e-06 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309317+0000", 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8920000000000001e-06 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329639+0000", 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020322030000000001 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329706+0000", 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6291000000000006e-05 2024-09-16T13:00:09.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329757+0000", 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1625000000000001e-05 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17079 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 663552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309319+0000", 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.298216166, 2024-09-16T13:00:09.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020674145000000001, 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17079 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309319+0000", 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309319+0000", 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309320+0000", 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3699999999999999e-07 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309320+0000", 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5099999999999998e-07 2024-09-16T13:00:09.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309323+0000", 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8270000000000002e-06 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309325+0000", 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8899999999999999e-06 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329805+0000", 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020479430999999999 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329901+0000", 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6507000000000003e-05 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.329993+0000", 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2301999999999998e-05 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:17080 4.2 4:43fa7829:::rbd_data.11b528bff1d7.0000000000000004:13 [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T12:59:51.309329+0000", 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.298206856, 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020787672, 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 17080 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.651 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309329+0000", 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309329+0000", 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309329+0000", 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3999999999999998e-07 2024-09-16T13:00:09.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309330+0000", 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2600000000000001e-07 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309333+0000", 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0410000000000002e-06 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.309335+0000", 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0169999999999999e-06 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.330016+0000", 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020681102999999999 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.330070+0000", 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4246999999999997e-05 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T12:59:51.330116+0000", 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6097999999999998e-05 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:39984 4.f 4:f4a00d3a:::rbd_data.11b59afb69bb.0000000000000032:16 [read 1253376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:00.304550+0000", 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3029858860000001, 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019087310999999999, 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 39984 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304550+0000", 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304550+0000", 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304550+0000", 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304552+0000", 2024-09-16T13:00:09.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2110000000000001e-06 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304559+0000", 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8739999999999997e-06 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304563+0000", 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1139999999999999e-06 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.323446+0000", 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018882258999999998 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.323546+0000", 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100893 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.323637+0000", 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0492e-05 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.659 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:39985 4.f 4:f4a00d3a:::rbd_data.11b59afb69bb.0000000000000032:16 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:00.304568+0000", 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3029678459999996, 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019343037, 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 39985 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304568+0000", 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304568+0000", 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304568+0000", 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3300000000000003e-07 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304570+0000", 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.08e-06 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304577+0000", 2024-09-16T13:00:09.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7070000000000003e-06 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.304581+0000", 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1320000000000004e-06 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.323687+0000", 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019105852999999999 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.323807+0000", 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120186 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.323911+0000", 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103646 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:39996 4.f 4:f4a00d3a:::rbd_data.11b59afb69bb.0000000000000032:16 [read 1200128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.664 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:00.306250+0000", 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3012850579999995, 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019078681, 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 39996 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306250+0000", 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306250+0000", 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306251+0000", 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8099999999999999e-07 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306252+0000", 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1799999999999999e-06 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306255+0000", 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4170000000000001e-06 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306257+0000", 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2019999999999998e-06 2024-09-16T13:00:09.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325118+0000", 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018860314999999999 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325228+0000", 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109992 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325329+0000", 2024-09-16T13:00:09.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101294 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:39997 4.f 4:f4a00d3a:::rbd_data.11b59afb69bb.0000000000000032:16 [read 1228800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:00.306260+0000", 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3012757550000007, 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019197800000000001, 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 39997 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306260+0000", 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306260+0000", 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306260+0000", 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.91e-07 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306261+0000", 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.207e-06 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306266+0000", 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0359999999999997e-06 2024-09-16T13:00:09.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306268+0000", 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1950000000000002e-06 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325365+0000", 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019096980999999999 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325413+0000", 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7998999999999999e-05 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325458+0000", 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4091000000000002e-05 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:39998 4.f 4:f4a00d3a:::rbd_data.11b59afb69bb.0000000000000032:16 [read 1257472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.673 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:00.306271+0000", 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3012645700000007, 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01932381, 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 39998 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306271+0000", 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306271+0000", 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306271+0000", 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0899999999999997e-07 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306273+0000", 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.347e-06 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306276+0000", 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4189999999999998e-06 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306279+0000", 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9579999999999999e-06 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325490+0000", 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019211097999999999 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325541+0000", 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1026000000000002e-05 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325595+0000", 2024-09-16T13:00:09.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3653000000000003e-05 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:39999 4.f 4:f4a00d3a:::rbd_data.11b59afb69bb.0000000000000032:16 [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:00.306283+0000", 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3012528369999998, 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01946788, 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 39999 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306283+0000", 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306283+0000", 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306283+0000", 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306284+0000", 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2020000000000001e-06 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306288+0000", 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4079999999999998e-06 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.306290+0000", 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2230000000000001e-06 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325636+0000", 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019346669 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325706+0000", 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9379000000000001e-05 2024-09-16T13:00:09.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.325751+0000", 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4709999999999997e-05 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:40012 4.f 4:f4a00d3a:::rbd_data.11b59afb69bb.0000000000000032:16 [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:00.308240+0000", 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.2992959420000005, 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019027941, 2024-09-16T13:00:09.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 40012 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308240+0000", 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308240+0000", 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308240+0000", 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4499999999999997e-07 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308242+0000", 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9980000000000002e-06 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308257+0000", 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5311e-05 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308260+0000", 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.185e-06 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.327172+0000", 2024-09-16T13:00:09.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018911174999999999 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.327225+0000", 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3813999999999997e-05 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.327267+0000", 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2012999999999999e-05 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:40013 4.f 4:f4a00d3a:::rbd_data.11b59afb69bb.0000000000000032:16 [read 1290240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:00.308265+0000", 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.2992701019999995, 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019125006999999999, 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 40013 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308265+0000", 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308265+0000", 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308266+0000", 2024-09-16T13:00:09.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8400000000000004e-07 2024-09-16T13:00:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308268+0000", 2024-09-16T13:00:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1780000000000002e-06 2024-09-16T13:00:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308275+0000", 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7429999999999996e-06 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.308279+0000", 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0759999999999996e-06 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.327289+0000", 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019009927999999999 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.327343+0000", 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3690999999999998e-05 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:00.327390+0000", 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7707000000000002e-05 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.691 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42631 4.2 4:417a7a72:::rbd_data.11b52dde2e.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.459242+0000", 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.148293056, 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037974714, 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42631 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.459242+0000", 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.459242+0000", 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.459245+0000", 2024-09-16T13:00:09.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9510000000000002e-06 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.459257+0000", 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1506e-05 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.459286+0000", 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.868e-05 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.459297+0000", 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1157e-05 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.459369+0000", 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2003999999999997e-05 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.459469+0000", 2024-09-16T13:00:09.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010038400000000001 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.466831+0000", 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.007361991 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.496965+0000", 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030134066000000001 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.497045+0000", 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9592000000000006e-05 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.497145+0000", 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100313 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.497217+0000", 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2070000000000006e-05 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44342 4.f 4:f1291512:::rbd_data.11b52dde2e.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:04.795198+0000", 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.812337758, 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024740944000000001, 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44342 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.795198+0000", 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.795198+0000", 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.795200+0000", 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1270000000000001e-06 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.795211+0000", 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0776e-05 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.795237+0000", 2024-09-16T13:00:09.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-05 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.795248+0000", 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1123e-05 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.795349+0000", 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010038400000000001 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.795507+0000", 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015800800000000001 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.800797+0000", 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052900919999999997 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.819817+0000", 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019020504000000001 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.819833+0000", 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5739999999999998e-05 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.819897+0000", 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3713999999999993e-05 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.819939+0000", 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1805999999999999e-05 2024-09-16T13:00:09.703 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:45048 4.2 4:417a7a72:::rbd_data.11b52dde2e.000000000000000a:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:06.385229+0000", 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.2223065879999999, 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029637882000000001, 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:09.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 45048 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.385229+0000", 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.385229+0000", 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.385231+0000", 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.677e-06 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.385238+0000", 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3459999999999996e-06 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.385255+0000", 2024-09-16T13:00:09.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6911000000000001e-05 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.385261+0000", 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.489e-06 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.385318+0000", 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6487999999999999e-05 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.385385+0000", 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7234e-05 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.390315+0000", 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049294960000000002 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.414686+0000", 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024371704000000001 2024-09-16T13:00:09.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.414703+0000", 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6826000000000001e-05 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.414800+0000", 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6547999999999998e-05 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:06.414867+0000", 2024-09-16T13:00:09.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7162999999999994e-05 2024-09-16T13:00:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:09.710 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:09.710 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:09.711 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:09.712 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:09.706+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:09.812 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:09.803+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:09.816 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:09.816 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:09.816 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:09.825 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:09.913 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:09.906+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:09.927 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:09.927 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:09.927 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:09.937 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:10.014 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:10.007+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:10.039 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42693 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.039 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612231+0000", 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4200940029999991, 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012032324, 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42693 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612231+0000", 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612231+0000", 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612233+0000", 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3599999999999999e-06 2024-09-16T13:00:10.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612240+0000", 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612257+0000", 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7079999999999999e-05 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612268+0000", 2024-09-16T13:00:10.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.077e-05 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.614188+0000", 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019201369999999999 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.614263+0000", 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4841999999999999e-05 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.614570+0000", 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030733900000000001 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.617786+0000", 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032162639999999999 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624164+0000", 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063774230000000001 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624222+0000", 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8595999999999999e-05 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624264+0000", 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1476000000000001e-05 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42694 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612323+0000", 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4200022699999995, 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012121108, 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42694 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612323+0000", 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612323+0000", 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612325+0000", 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5e-06 2024-09-16T13:00:10.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612334+0000", 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2329999999999995e-06 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612354+0000", 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0007e-05 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612365+0000", 2024-09-16T13:00:10.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.115e-05 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.615722+0000", 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033573029999999998 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.615795+0000", 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2933000000000001e-05 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.616132+0000", 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033686099999999999 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.619019+0000", 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028870829999999999 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624319+0000", 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052999320000000003 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624399+0000", 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0051999999999999e-05 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624444+0000", 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5053999999999999e-05 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42695 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612378+0000", 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4199476539999996, 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012263121, 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42695 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612378+0000", 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612378+0000", 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612379+0000", 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.144e-06 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612386+0000", 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2130000000000002e-06 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612406+0000", 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9797e-05 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612416+0000", 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0516000000000001e-05 2024-09-16T13:00:10.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.616598+0000", 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041817080000000001 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.616645+0000", 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6604999999999998e-05 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.616832+0000", 2024-09-16T13:00:10.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018746100000000001 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.620343+0000", 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035106730000000002 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624542+0000", 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041995469999999997 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624599+0000", 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6453000000000003e-05 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624641+0000", 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2004e-05 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42696 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.057 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612428+0000", 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4198973769999998, 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012368262, 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42696 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612428+0000", 2024-09-16T13:00:10.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612428+0000", 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612429+0000", 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.093e-06 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612436+0000", 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7440000000000001e-06 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612452+0000", 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6030000000000001e-05 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612462+0000", 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.617243+0000", 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004781062 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.617289+0000", 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6387999999999997e-05 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.617484+0000", 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019473799999999999 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.620350+0000", 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028657079999999998 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624712+0000", 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043620480000000003 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624762+0000", 2024-09-16T13:00:10.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9818000000000003e-05 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624796+0000", 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4567000000000003e-05 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42697 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612473+0000", 2024-09-16T13:00:10.063 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4198521940000006, 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012554299, 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42697 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612473+0000", 2024-09-16T13:00:10.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612473+0000", 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612474+0000", 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1340000000000001e-06 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612481+0000", 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4559999999999997e-06 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612497+0000", 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6637e-05 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612507+0000", 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5230000000000001e-06 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.617817+0000", 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0053100880000000001 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.617861+0000", 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4057000000000002e-05 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.618071+0000", 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021045600000000001 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.622041+0000", 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003969803 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624844+0000", 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028023169999999999 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624928+0000", 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3975999999999997e-05 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.625027+0000", 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9852000000000006e-05 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42698 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.069 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612523+0000", 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4198022510000001, 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013443225, 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42698 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612523+0000", 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612523+0000", 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612524+0000", 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.08e-06 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612531+0000", 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6800000000000004e-06 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612547+0000", 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6626999999999999e-05 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612557+0000", 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2229999999999996e-06 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.618455+0000", 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0058981900000000002 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.618499+0000", 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4166999999999999e-05 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.618693+0000", 2024-09-16T13:00:10.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019395999999999999 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.622044+0000", 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033509249999999998 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.625886+0000", 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038419270000000002 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.625932+0000", 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6656000000000002e-05 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.625966+0000", 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.379e-05 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42699 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.075 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612568+0000", 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4197577779999992, 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013499404, 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42699 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612568+0000", 2024-09-16T13:00:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612568+0000", 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612569+0000", 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0139999999999999e-06 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612575+0000", 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1700000000000002e-06 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612591+0000", 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6489000000000001e-05 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612601+0000", 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3170000000000008e-06 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.619051+0000", 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0064508289999999999 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.619095+0000", 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4082999999999998e-05 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.619284+0000", 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018863899999999999 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624073+0000", 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047889789999999996 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.626000+0000", 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001927361 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.626037+0000", 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6541000000000001e-05 2024-09-16T13:00:10.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.626067+0000", 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9981999999999999e-05 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42700 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612611+0000", 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4197139980000006, 2024-09-16T13:00:10.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013555472000000001, 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42700 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612611+0000", 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612611+0000", 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612612+0000", 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0100000000000001e-06 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612619+0000", 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3169999999999998e-06 2024-09-16T13:00:10.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612635+0000", 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6002999999999999e-05 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612646+0000", 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1793e-05 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.619653+0000", 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070069709999999999 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.619721+0000", 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7213999999999997e-05 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.619956+0000", 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000235474 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624083+0000", 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004126869 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.626100+0000", 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020171619999999999 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.626136+0000", 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5889000000000001e-05 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.626167+0000", 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0769999999999998e-05 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42701 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612662+0000", 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4196634980000002, 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013614342, 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42701 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612662+0000", 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612662+0000", 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612663+0000", 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0869999999999999e-06 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612669+0000", 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6100000000000002e-06 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612686+0000", 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6506000000000001e-05 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612695+0000", 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3340000000000008e-06 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.620496+0000", 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0078008720000000004 2024-09-16T13:00:10.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.620570+0000", 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3771999999999998e-05 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.620911+0000", 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034125099999999999 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624096+0000", 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031843599999999998 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.626204+0000", 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021087290000000002 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.626246+0000", 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1359999999999997e-05 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.626276+0000", 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0460999999999999e-05 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42702 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612733+0000", 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4195923280000002, 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014422671999999999, 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42702 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612733+0000", 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612733+0000", 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612734+0000", 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.513e-06 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612742+0000", 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5970000000000003e-06 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612761+0000", 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8940000000000002e-05 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612771+0000", 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0263e-05 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.621341+0000", 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085694650000000001 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.621398+0000", 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7729000000000002e-05 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.621668+0000", 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026999800000000003 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624655+0000", 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029866430000000002 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.627026+0000", 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002370792 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.627118+0000", 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2095999999999994e-05 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.627156+0000", 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7635999999999997e-05 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:42703 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:01.612783+0000", 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4195424320000001, 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015030781999999999, 2024-09-16T13:00:10.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 42703 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612783+0000", 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612783+0000", 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612784+0000", 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0139999999999999e-06 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612790+0000", 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2230000000000004e-06 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612811+0000", 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.105e-05 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.612821+0000", 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4360000000000008e-06 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.622091+0000", 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0092702389999999996 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.622177+0000", 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6285000000000005e-05 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.622557+0000", 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038019900000000001 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.624662+0000", 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00210516 2024-09-16T13:00:10.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.627724+0000", 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030616810000000001 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.627781+0000", 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7123999999999997e-05 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:01.627814+0000", 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2370999999999999e-05 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44294 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:04.688304+0000", 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3440208890000003, 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011602042, 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44294 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688304+0000", 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688304+0000", 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688306+0000", 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6470000000000001e-06 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688315+0000", 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2620000000000004e-06 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688356+0000", 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0850999999999999e-05 2024-09-16T13:00:10.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688367+0000", 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0723999999999999e-05 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.695889+0000", 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075225250000000004 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.696021+0000", 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013127599999999999 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.696797+0000", 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077656399999999999 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.698997+0000", 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002199866 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.699778+0000", 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00078063600000000003 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.699862+0000", 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4320000000000006e-05 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.699906+0000", 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4370999999999999e-05 2024-09-16T13:00:10.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44295 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:04.688382+0000", 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3439432419999999, 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012954174000000001, 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44295 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688382+0000", 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688382+0000", 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688383+0000", 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688389+0000", 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8819999999999998e-06 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688414+0000", 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4995e-05 2024-09-16T13:00:10.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.688422+0000", 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0269999999999995e-06 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.697848+0000", 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0094258069999999996 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.697993+0000", 2024-09-16T13:00:10.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014494200000000001 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.698252+0000", 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025964599999999999 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.701259+0000", 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030069459999999999 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.701270+0000", 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0676000000000001e-05 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.701309+0000", 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9144000000000003e-05 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:04.701336+0000", 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7146999999999999e-05 2024-09-16T13:00:10.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44710 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:05.779527+0000", 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2527983069999999, 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011056593999999999, 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44710 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779527+0000", 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779527+0000", 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779528+0000", 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4499999999999996e-07 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779533+0000", 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0749999999999997e-06 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779544+0000", 2024-09-16T13:00:10.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0784999999999999e-05 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779549+0000", 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3979999999999998e-06 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.787118+0000", 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075692279999999999 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.787181+0000", 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2879999999999994e-05 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.787563+0000", 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038154500000000001 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.790420+0000", 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028577400000000001 2024-09-16T13:00:10.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.790428+0000", 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9000000000000006e-06 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.790534+0000", 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010574 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.790584+0000", 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9458e-05 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44711 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:05.779554+0000", 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2527711090000002, 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016538047, 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44711 2024-09-16T13:00:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779554+0000", 2024-09-16T13:00:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779554+0000", 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779555+0000", 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9200000000000001e-07 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779558+0000", 2024-09-16T13:00:10.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.54e-06 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779565+0000", 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8600000000000004e-06 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779569+0000", 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.267e-06 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.788983+0000", 2024-09-16T13:00:10.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0094138659999999999 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.789074+0000", 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0688999999999999e-05 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.789770+0000", 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00069649800000000004 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.792905+0000", 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00313445 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.795996+0000", 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030916580000000002 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.796052+0000", 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5325000000000003e-05 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.796092+0000", 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0401999999999997e-05 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44712 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:05.779574+0000", 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2527513780000001, 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017398382, 2024-09-16T13:00:10.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44712 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779574+0000", 2024-09-16T13:00:10.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779574+0000", 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779574+0000", 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-16T13:00:10.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779578+0000", 2024-09-16T13:00:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4199999999999999e-06 2024-09-16T13:00:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779584+0000", 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5080000000000002e-06 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779588+0000", 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9689999999999996e-06 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.790626+0000", 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011037867999999999 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.790706+0000", 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9875000000000004e-05 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.791073+0000", 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036726 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.792954+0000", 2024-09-16T13:00:10.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018810249999999999 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.796262+0000", 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033075349999999999 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.796923+0000", 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00066126200000000003 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.796972+0000", 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9197999999999999e-05 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44713 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:05.779593+0000", 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2527323849999998, 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017495849000000001, 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44713 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779593+0000", 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779593+0000", 2024-09-16T13:00:10.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779593+0000", 2024-09-16T13:00:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8599999999999999e-07 2024-09-16T13:00:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779597+0000", 2024-09-16T13:00:10.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4319999999999999e-06 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779603+0000", 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3740000000000003e-06 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.779607+0000", 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.791718+0000", 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012110798000000001 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.791810+0000", 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2553999999999995e-05 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.792273+0000", 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00046301300000000002 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.796170+0000", 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003896428 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.796989+0000", 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00081966500000000002 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.797043+0000", 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3692e-05 2024-09-16T13:00:10.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.797089+0000", 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5716000000000001e-05 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44813 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:05.940648+0000", 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0916776119999998, 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011055555, 2024-09-16T13:00:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44813 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940648+0000", 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940648+0000", 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940648+0000", 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7899999999999997e-07 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940652+0000", 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7809999999999999e-06 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940668+0000", 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6005999999999999e-05 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940676+0000", 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1510000000000004e-06 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.945894+0000", 2024-09-16T13:00:10.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052181010000000002 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.946011+0000", 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011620000000000001 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.946895+0000", 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088467500000000002 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.949508+0000", 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026125079999999999 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.951606+0000", 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002098319 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.951660+0000", 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3801999999999998e-05 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.951703+0000", 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3232999999999998e-05 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44814 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:05.940682+0000", 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0916433100000003, 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012277704, 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44814 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940682+0000", 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940682+0000", 2024-09-16T13:00:10.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940683+0000", 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1200000000000003e-07 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940686+0000", 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6150000000000001e-06 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940696+0000", 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940701+0000", 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6330000000000004e-06 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.948438+0000", 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077368699999999999 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.948549+0000", 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111531 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.948841+0000", 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000291525 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.951562+0000", 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0027212759999999999 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.952882+0000", 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001319703 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.952923+0000", 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1736999999999997e-05 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.952960+0000", 2024-09-16T13:00:10.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6402000000000001e-05 2024-09-16T13:00:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:44815 4.1c 4:3e0dc48f:::rbd_object_map.11b52dde2e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T13:00:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:05.940705+0000", 2024-09-16T13:00:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0916200229999999, 2024-09-16T13:00:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013131983999999999, 2024-09-16T13:00:10.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 44815 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940705+0000", 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940705+0000", 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940706+0000", 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5000000000000003e-07 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940709+0000", 2024-09-16T13:00:10.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1779999999999999e-06 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940726+0000", 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7450999999999999e-05 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.940741+0000", 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4491000000000001e-05 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.949529+0000", 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087878139999999997 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.949612+0000", 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3486000000000007e-05 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.949970+0000", 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000357382 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.951713+0000", 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001743414 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.953771+0000", 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020576570000000001 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.953800+0000", 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9771e-05 2024-09-16T13:00:10.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:05.953837+0000", 2024-09-16T13:00:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6789999999999998e-05 2024-09-16T13:00:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.162 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.162 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:10.164 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:10.164 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:10.158+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:10.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T13:00:10.265 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:10.258+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:10.288 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:10.288 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:10.288 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:10.299 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:10.301 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone21 2024-09-16T13:00:10.366 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:10.358+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:10.405 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:10.405 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:10.405 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:10.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:10.466 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:10.459+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:10.521 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47613 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3571712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15742446700000001, 2024-09-16T13:00:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054671112000000001, 2024-09-16T13:00:10.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47613 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5699999999999999e-07 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357529+0000", 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1459999999999999e-06 2024-09-16T13:00:10.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357532+0000", 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357534+0000", 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9089999999999998e-06 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412120+0000", 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054585897000000001 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412159+0000", 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8967000000000001e-05 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412199+0000", 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0068999999999999e-05 2024-09-16T13:00:10.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47614 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3588096~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357536+0000", 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15741577000000001, 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054770905000000002, 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47614 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357536+0000", 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357536+0000", 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357537+0000", 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357538+0000", 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0950000000000001e-06 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357541+0000", 2024-09-16T13:00:10.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9979999999999999e-06 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357543+0000", 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9539999999999998e-06 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412227+0000", 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054684501000000003 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412266+0000", 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8764999999999999e-05 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412307+0000", 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1341000000000003e-05 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.530 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47615 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3604480~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.157407465, 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054886719, 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47615 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.41e-07 2024-09-16T13:00:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357546+0000", 2024-09-16T13:00:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.099e-06 2024-09-16T13:00:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357549+0000", 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8820000000000001e-06 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357551+0000", 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.852e-06 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412335+0000", 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054784294999999997 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412392+0000", 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6938000000000003e-05 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412431+0000", 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47616 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3620864~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.157399386, 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054998867, 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47616 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4900000000000002e-07 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357554+0000", 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0669999999999999e-06 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357557+0000", 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9079999999999999e-06 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357559+0000", 2024-09-16T13:00:10.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9580000000000001e-06 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412469+0000", 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054909873999999997 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412512+0000", 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3430999999999998e-05 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412552+0000", 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9379999999999999e-05 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47617 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3637248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.157391222, 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055098616000000003, 2024-09-16T13:00:10.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47617 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6e-07 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357562+0000", 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0809999999999999e-06 2024-09-16T13:00:10.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357565+0000", 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357567+0000", 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412578+0000", 2024-09-16T13:00:10.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055010816999999997 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412615+0000", 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7629999999999997e-05 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412660+0000", 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4116999999999998e-05 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47618 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3653632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15738191800000001, 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055192228000000003, 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47618 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.545 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:10.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3099999999999999e-07 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357573+0000", 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9919999999999999e-06 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357577+0000", 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.208e-06 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357579+0000", 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412685+0000", 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055105981999999998 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412723+0000", 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8028999999999998e-05 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412762+0000", 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9221999999999998e-05 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47619 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3670016~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357581+0000", 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15737076999999999, 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055287718999999999, 2024-09-16T13:00:10.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47619 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357581+0000", 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357581+0000", 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357582+0000", 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6100000000000002e-07 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357583+0000", 2024-09-16T13:00:10.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.085e-06 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357586+0000", 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.931e-06 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357588+0000", 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412788+0000", 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055200013999999999 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412827+0000", 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9002999999999999e-05 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412869+0000", 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2546000000000002e-05 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47620 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3686400~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357590+0000", 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15736183200000001, 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0553897, 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47620 2024-09-16T13:00:10.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357590+0000", 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357590+0000", 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357591+0000", 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357592+0000", 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.06e-06 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357595+0000", 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.999e-06 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357597+0000", 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9640000000000002e-06 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412900+0000", 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055303123000000003 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412941+0000", 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0907e-05 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412980+0000", 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47621 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3702784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357598+0000", 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15735375700000001, 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055490600000000001, 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47621 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357598+0000", 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357598+0000", 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357599+0000", 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5199999999999998e-07 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357600+0000", 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357603+0000", 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9560000000000002e-06 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357606+0000", 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1300000000000001e-06 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413012+0000", 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055406186000000003 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413051+0000", 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8686999999999997e-05 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413089+0000", 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8223000000000003e-05 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47622 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3719168~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15734420399999999, 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055581815, 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47622 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4699999999999998e-07 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357609+0000", 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7399999999999991e-07 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357612+0000", 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9799999999999998e-06 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357615+0000", 2024-09-16T13:00:10.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.269e-06 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413115+0000", 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055499522000000003 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413154+0000", 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8584999999999997e-05 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413190+0000", 2024-09-16T13:00:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6238000000000001e-05 2024-09-16T13:00:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47623 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357617+0000", 2024-09-16T13:00:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.157334748, 2024-09-16T13:00:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055603054999999998, 2024-09-16T13:00:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47623 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357617+0000", 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357617+0000", 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357618+0000", 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6199999999999999e-07 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357619+0000", 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.032e-06 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357622+0000", 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9679999999999998e-06 2024-09-16T13:00:10.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357624+0000", 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413215+0000", 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055591451 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413220+0000", 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4620000000000004e-06 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47624 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357625+0000", 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.157326735, 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055617605000000001, 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47624 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357625+0000", 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357625+0000", 2024-09-16T13:00:10.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357626+0000", 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4900000000000002e-07 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357627+0000", 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2139999999999999e-06 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357630+0000", 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9409999999999999e-06 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357633+0000", 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6960000000000001e-06 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413236+0000", 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055603303999999999 2024-09-16T13:00:10.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413243+0000", 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2010000000000002e-06 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47625 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.157316177, 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055628134000000003, 2024-09-16T13:00:10.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47625 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357637+0000", 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5300000000000002e-07 2024-09-16T13:00:10.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357640+0000", 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357642+0000", 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9130000000000001e-06 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413257+0000", 2024-09-16T13:00:10.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055615255000000002 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413264+0000", 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.849e-06 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47626 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:10.580 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15730819100000001, 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055641412000000001, 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47626 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5600000000000002e-07 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357645+0000", 2024-09-16T13:00:10.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-06 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357648+0000", 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9859999999999999e-06 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357650+0000", 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.922e-06 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413278+0000", 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055628089999999998 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413285+0000", 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1289999999999998e-06 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47627 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:10.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15730002000000001, 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055653995999999997, 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47627 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357654+0000", 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.212e-06 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357656+0000", 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.897e-06 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357658+0000", 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8920000000000001e-06 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413299+0000", 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055640991000000001 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413306+0000", 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47628 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357660+0000", 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15729182799999999, 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055666457000000003, 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47628 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357660+0000", 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357660+0000", 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357661+0000", 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5499999999999999e-07 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357662+0000", 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.065e-06 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357665+0000", 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357668+0000", 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-16T13:00:10.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413320+0000", 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055652245000000003 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413327+0000", 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7989999999999996e-06 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47629 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357670+0000", 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15728191899999999, 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055676011999999997, 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47629 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.593 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357670+0000", 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357670+0000", 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357671+0000", 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4699999999999998e-07 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357672+0000", 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.015e-06 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357675+0000", 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0359999999999998e-06 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357677+0000", 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.481e-06 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413340+0000", 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662752000000003 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413346+0000", 2024-09-16T13:00:10.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4810000000000003e-06 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47630 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.157273155, 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055688423000000001, 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:10.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47630 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:10.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357680+0000", 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.11e-06 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357683+0000", 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1250000000000001e-06 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357686+0000", 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-06 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413360+0000", 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055674003 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413367+0000", 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9189999999999999e-06 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47631 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357688+0000", 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15726464900000001, 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055700309000000003, 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:10.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47631 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357688+0000", 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357688+0000", 2024-09-16T13:00:10.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357733+0000", 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5716000000000001e-05 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357740+0000", 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3740000000000003e-06 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357748+0000", 2024-09-16T13:00:10.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4919999999999993e-06 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357753+0000", 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1259999999999997e-06 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413381+0000", 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055627958999999998 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413388+0000", 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47632 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357759+0000", 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.15719317299999999, 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055652716999999997, 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:10.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47632 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357759+0000", 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357759+0000", 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359561+0000", 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018016060000000001 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359564+0000", 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359577+0000", 2024-09-16T13:00:10.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3087e-05 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359582+0000", 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2549999999999999e-06 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413404+0000", 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.053822163999999999 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:10.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413412+0000", 2024-09-16T13:00:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0330000000000008e-06 2024-09-16T13:00:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:10.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:10.609 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:10.611 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:00:10.611 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:10.606+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:10.653 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-16T13:00:10.712 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:10.706+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:10.721 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:10.721 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:10.730 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:00:10.812 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:10.806+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:10.825 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:10.825 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:10.825 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:10.825 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:10.825 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:10.835 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:00:10.894 INFO:teuthology.orchestra.run.smithi042.stdout:4016 trunc from 0xdf1d065 to 0x8131353 2024-09-16T13:00:10.913 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:10.906+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:10.925 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:10.925 INFO:teuthology.orchestra.run.smithi042.stdout:4018 punch from 0x8129898 to 0x8131353, (0x7abb bytes) 2024-09-16T13:00:10.925 INFO:teuthology.orchestra.run.smithi042.stdout:4020 punch from 0x167d081 to 0x1683142, (0x60c1 bytes) 2024-09-16T13:00:10.938 INFO:teuthology.orchestra.run.smithi042.stdout:4022 trunc from 0x8131353 to 0x10b1635 2024-09-16T13:00:10.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:11.000 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:590 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.656372408999999, 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034258392999999998, 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 590 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339271+0000", 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339281+0000", 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6670000000000008e-06 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339307+0000", 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5907000000000001e-05 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339317+0000", 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0356999999999999e-05 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339412+0000", 2024-09-16T13:00:11.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4248999999999998e-05 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339533+0000", 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012184900000000001 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.347489+0000", 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079559599999999998 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373423+0000", 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025933971 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373435+0000", 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1924e-05 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373492+0000", 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6842999999999998e-05 2024-09-16T13:00:11.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373528+0000", 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5063 4.13 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22711298~51120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.686227611, 2024-09-16T13:00:11.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111447293, 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5063 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:11.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309416+0000", 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358861+0000", 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049444717999999999 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358893+0000", 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2279e-05 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358902+0000", 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4390000000000006e-06 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359062+0000", 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016014100000000001 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359330+0000", 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026734699999999997 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.362566+0000", 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032358769999999999 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.417541+0000", 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054975345000000002 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420706+0000", 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031651380000000001 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420790+0000", 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4283999999999995e-05 2024-09-16T13:00:11.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420862+0000", 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1138999999999998e-05 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5225 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.191072290999998, 2024-09-16T13:00:11.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027003329999999999, 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5225 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804571+0000", 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6309999999999999e-06 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804579+0000", 2024-09-16T13:00:11.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804600+0000", 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1114000000000001e-05 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804613+0000", 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3040000000000001e-05 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804713+0000", 2024-09-16T13:00:11.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9854999999999999e-05 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804844+0000", 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130963 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.810955+0000", 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061105880000000001 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831495+0000", 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020540467 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831507+0000", 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831544+0000", 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7413000000000002e-05 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831573+0000", 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8836e-05 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5504 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.527368684999999, 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020881448, 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.019 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5504 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468276+0000", 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8370000000000001e-06 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468290+0000", 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.383e-05 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468327+0000", 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468341+0000", 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.331e-05 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468439+0000", 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8746999999999995e-05 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468593+0000", 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015343999999999999 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.472806+0000", 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042129849999999998 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488823+0000", 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016017686999999999 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488936+0000", 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112074 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489029+0000", 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3801000000000005e-05 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489155+0000", 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125248 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8354 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.714274615000001, 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016884742000000001, 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8354 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281370+0000", 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3159999999999999e-06 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281382+0000", 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281413+0000", 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1519000000000001e-05 2024-09-16T13:00:11.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281435+0000", 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1812999999999999e-05 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292521+0000", 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011085915999999999 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292628+0000", 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107311 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293108+0000", 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047976600000000001 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297972+0000", 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048640200000000001 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298174+0000", 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020222 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298226+0000", 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2058000000000002e-05 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298252+0000", 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5743000000000001e-05 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.030 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8677 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.985683645000002, 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01642155, 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8677 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009959+0000", 2024-09-16T13:00:11.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2899999999999999e-07 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009961+0000", 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3760000000000002e-06 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009965+0000", 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009968+0000", 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1190000000000001e-06 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023570+0000", 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013601317999999999 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023695+0000", 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012563400000000001 2024-09-16T13:00:11.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024191+0000", 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000495434 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026285+0000", 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020948360000000001 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026295+0000", 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8889999999999997e-06 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026338+0000", 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2253999999999999e-05 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026380+0000", 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2153000000000001e-05 2024-09-16T13:00:11.036 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8678 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.985669393000002, 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020138508999999999, 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.037 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8678 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009973+0000", 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3000000000000002e-07 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009975+0000", 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4739999999999999e-06 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009980+0000", 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4399999999999998e-06 2024-09-16T13:00:11.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009983+0000", 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1209999999999998e-06 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025464+0000", 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015480918 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025536+0000", 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2212000000000006e-05 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025768+0000", 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000232186 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029964+0000", 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041957899999999996 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029981+0000", 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6634e-05 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030070+0000", 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9487999999999994e-05 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030111+0000", 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0915999999999999e-05 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8718 4.1b 4:dbb4ad27:::rbd_data.11b55eb65e78.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.909354432000001, 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028260378999999999, 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8718 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086290+0000", 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1950000000000002e-06 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086300+0000", 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0458999999999999e-05 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086331+0000", 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1117e-05 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086344+0000", 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3191e-05 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086423+0000", 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8288999999999995e-05 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086530+0000", 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010720799999999999 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.091612+0000", 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050817429999999997 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114274+0000", 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022662832000000001 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114386+0000", 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011155899999999999 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114475+0000", 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9043999999999997e-05 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114548+0000", 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2742000000000003e-05 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.048 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24294 4.1b 4:d9b96624:::rbd_data.11b59afb69bb.0000000000000000:16 [read 1413120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.525030936, 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016422685999999999, 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24294 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.049 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5499999999999999e-07 2024-09-16T13:00:11.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470612+0000", 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1790000000000001e-06 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470615+0000", 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0580000000000002e-06 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470618+0000", 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.103e-06 2024-09-16T13:00:11.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486878+0000", 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016260398999999998 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486954+0000", 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5763e-05 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487034+0000", 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9929e-05 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24328 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1339392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.524855079, 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016079547999999999, 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:11.053 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24328 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0699999999999998e-07 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470789+0000", 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9659999999999999e-06 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470795+0000", 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4110000000000004e-06 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470798+0000", 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5499999999999999e-06 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486710+0000", 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015911536 2024-09-16T13:00:11.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486790+0000", 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9912000000000003e-05 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486866+0000", 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6766000000000003e-05 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.057 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24329 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1368064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.524829841000001, 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016167826999999999, 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24329 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:11.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9999999999999998e-07 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470815+0000", 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6900000000000001e-06 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470821+0000", 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9050000000000002e-06 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470823+0000", 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:00:11.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486891+0000", 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016068000999999998 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486933+0000", 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1711000000000001e-05 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486980+0000", 2024-09-16T13:00:11.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6643e-05 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24330 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.524816230999999, 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016312199999999999, 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.062 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24330 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6199999999999999e-07 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470827+0000", 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.333e-06 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470831+0000", 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.348e-06 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470832+0000", 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.849e-06 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486993+0000", 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016160834999999998 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487060+0000", 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6748000000000005e-05 2024-09-16T13:00:11.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487138+0000", 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7824999999999994e-05 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24331 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1425408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.524807534000001, 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016546297000000001, 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.066 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24331 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470835+0000", 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4999999999999999e-07 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470836+0000", 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2869999999999999e-06 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470839+0000", 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0479999999999999e-06 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470841+0000", 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0090000000000002e-06 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487212+0000", 2024-09-16T13:00:11.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016370609000000001 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487288+0000", 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6584999999999999e-05 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487381+0000", 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2509000000000005e-05 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24358 4.1b 4:d9b96624:::rbd_data.11b59afb69bb.0000000000000000:16 [read 1417216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.523626821000001, 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015994596, 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24358 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472016+0000", 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5199999999999996e-07 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472017+0000", 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4249999999999999e-06 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472021+0000", 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.596e-06 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472024+0000", 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5860000000000001e-06 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487784+0000", 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015760066999999999 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487912+0000", 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012761900000000001 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488010+0000", 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7651000000000004e-05 2024-09-16T13:00:11.074 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24393 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1372160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.523327087, 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016028144000000001, 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.075 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24393 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9900000000000001e-07 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472317+0000", 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.547e-06 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472321+0000", 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8970000000000001e-06 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472325+0000", 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7759999999999997e-06 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488206+0000", 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015881025 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488272+0000", 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5982000000000006e-05 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488343+0000", 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0518000000000005e-05 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24394 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:11.079 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.523313738000001, 2024-09-16T13:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016119201, 2024-09-16T13:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24394 2024-09-16T13:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.080 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472329+0000", 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9500000000000003e-07 2024-09-16T13:00:11.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472330+0000", 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6360000000000001e-06 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472334+0000", 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7730000000000001e-06 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472337+0000", 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7420000000000002e-06 2024-09-16T13:00:11.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488362+0000", 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016025094 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488405+0000", 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3438e-05 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488447+0000", 2024-09-16T13:00:11.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2023000000000001e-05 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24395 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1429504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.523302597000001, 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016957005000000001, 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24395 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472340+0000", 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2099999999999997e-07 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472341+0000", 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4190000000000001e-06 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472345+0000", 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8139999999999998e-06 2024-09-16T13:00:11.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472348+0000", 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5780000000000001e-06 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489143+0000", 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016795244000000001 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489213+0000", 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0438000000000006e-05 2024-09-16T13:00:11.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489296+0000", 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2991000000000006e-05 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:42621 4.e 4:744fcf8a:::rbd_data.11b52dde2e.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:01.415676+0000", 2024-09-16T13:00:11.088 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5799660109999998, 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034249963000000001, 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 42621 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415676+0000", 2024-09-16T13:00:11.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415676+0000", 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415677+0000", 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.564e-06 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415685+0000", 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9060000000000002e-06 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415710+0000", 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.421e-05 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415718+0000", 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7930000000000003e-06 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415779+0000", 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0191e-05 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415887+0000", 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108936 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.423612+0000", 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077250340000000004 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449730+0000", 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026117745000000001 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449792+0000", 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2277e-05 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449860+0000", 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7794999999999997e-05 2024-09-16T13:00:11.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449926+0000", 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5512000000000006e-05 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:44349 4.e 4:744fcf8a:::rbd_data.11b52dde2e.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:04.840761+0000", 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.154881305, 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020098765000000001, 2024-09-16T13:00:11.094 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44349 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840761+0000", 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840761+0000", 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840764+0000", 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3330000000000001e-06 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840778+0000", 2024-09-16T13:00:11.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3923e-05 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840835+0000", 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6994999999999999e-05 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840849+0000", 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4544e-05 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840972+0000", 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122707 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.841125+0000", 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000152867 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.847862+0000", 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006737219 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.860451+0000", 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012589352999999999 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.860582+0000", 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130061 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.860739+0000", 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000157823 2024-09-16T13:00:11.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.860859+0000", 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011993999999999999 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:44403 4.e 4:744fcf8a:::rbd_data.11b52dde2e.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:04.981408+0000", 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0142338950000003, 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036194363, 2024-09-16T13:00:11.100 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44403 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981408+0000", 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981408+0000", 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981410+0000", 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9470000000000002e-06 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981419+0000", 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5200000000000003e-06 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981443+0000", 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3085999999999999e-05 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981469+0000", 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6573000000000001e-05 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981596+0000", 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012663400000000001 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:11.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981740+0000", 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000144036 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.990365+0000", 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086247619999999994 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:05.017381+0000", 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027016209999999999 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:11.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:05.017406+0000", 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5318999999999999e-05 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:05.017530+0000", 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012376600000000001 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:05.017602+0000", 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2509999999999995e-05 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.105 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.106 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:11.106 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:11.106 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:11.107 INFO:teuthology.orchestra.run.smithi042.stdout:4024 read 0x21a3f7 thru 0x22174a (0x7354 bytes) 2024-09-16T13:00:11.107 INFO:teuthology.orchestra.run.smithi042.stdout:4025 write 0x197a4d0 thru 0x197d2ad (0x2dde bytes) 2024-09-16T13:00:11.107 INFO:teuthology.orchestra.run.smithi042.stdout:4026 write 0x1fe6f54 thru 0x1febc9d (0x4d4a bytes) 2024-09-16T13:00:11.107 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:11.108 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:11.108 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:11.102+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:11.109 INFO:teuthology.orchestra.run.smithi042.stdout:4027 writesame 0x85c130b thru 0x85c36b0 data_size 0x1b(0x23a6 bytes) 2024-09-16T13:00:11.109 INFO:teuthology.orchestra.run.smithi042.stdout:4028 write 0x8af5f81 thru 0x8b05bd7 (0xfc57 bytes) 2024-09-16T13:00:11.109 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:11.109 INFO:teuthology.orchestra.run.smithi042.stdout:4029 punch from 0x89ad1e9 to 0x89b9226, (0xc03d bytes) 2024-09-16T13:00:11.109 INFO:teuthology.orchestra.run.smithi042.stdout:4030 punch from 0x40f6a7a to 0x41013f3, (0xa979 bytes) 2024-09-16T13:00:11.109 INFO:teuthology.orchestra.run.smithi042.stdout:4031 read 0x39ad58b thru 0x39b7148 (0x9bbe bytes) 2024-09-16T13:00:11.113 INFO:teuthology.orchestra.run.smithi042.stdout:4033 trunc from 0x8b05bd8 to 0xdf4825c 2024-09-16T13:00:11.122 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:11.124 INFO:teuthology.orchestra.run.smithi042.stdout:4038 read 0x2dd787d thru 0x2ddca36 (0x51ba bytes) 2024-09-16T13:00:11.136 INFO:teuthology.orchestra.run.smithi042.stdout:4040 trunc from 0xdf4825c to 0x5f59fd0 2024-09-16T13:00:11.140 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:11.156 INFO:teuthology.orchestra.run.smithi042.stdout:4041 read 0x4cd461e thru 0x4cd863f (0x4022 bytes) 2024-09-16T13:00:11.156 INFO:teuthology.orchestra.run.smithi042.stdout:4042 read 0x2034f9f thru 0x2041ca9 (0xcd0b bytes) 2024-09-16T13:00:11.165 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:00:11.168 INFO:teuthology.orchestra.run.smithi042.stdout:4044 write 0x67f6e78 thru 0x67f72a7 (0x430 bytes) 2024-09-16T13:00:11.177 INFO:teuthology.orchestra.run.smithi042.stdout:4046 trunc from 0x67f72a8 to 0x87423fa 2024-09-16T13:00:11.186 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:11.207 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T13:00:11.208 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:11.202+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:11.210 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:11.212 INFO:teuthology.orchestra.run.smithi042.stdout:4049 write 0xb32423c thru 0xb32f170 (0xaf35 bytes) 2024-09-16T13:00:11.221 INFO:teuthology.orchestra.run.smithi042.stdout:4050 read 0xa46596b thru 0xa472249 (0xc8df bytes) 2024-09-16T13:00:11.221 INFO:teuthology.orchestra.run.smithi042.stdout:4051 punch from 0xc359ed to 0xc3f6d3, (0x9ce6 bytes) 2024-09-16T13:00:11.221 INFO:teuthology.orchestra.run.smithi042.stdout:4052 write 0x82cf9b5 thru 0x82ddac4 (0xe110 bytes) 2024-09-16T13:00:11.233 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:11.236 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:11.236 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:11.236 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:11.236 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:11.236 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:00:11.237 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:11.237 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:11.237 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:11.241 INFO:teuthology.orchestra.run.smithi042.stdout:4054 write 0xd235e03 thru 0xd239a74 (0x3c72 bytes) 2024-09-16T13:00:11.248 INFO:teuthology.orchestra.run.smithi042.stdout:4056 writesame 0x1a4dc83 thru 0x1a550d4 data_size 0x15(0x7452 bytes) 2024-09-16T13:00:11.263 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:11.263 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:11.269 INFO:teuthology.orchestra.run.smithi042.stdout:4057 writesame 0xe1af826 thru 0xe1b26fc data_size 0x7(0x2ed7 bytes) 2024-09-16T13:00:11.269 INFO:teuthology.orchestra.run.smithi042.stdout:4058 write 0xcf60c6e thru 0xcf61a98 (0xe2b bytes) 2024-09-16T13:00:11.273 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:11.295 INFO:teuthology.orchestra.run.smithi042.stdout:4060 trunc from 0xe1b26fd to 0x3585f4f 2024-09-16T13:00:11.308 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:11.299+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:11.343 INFO:teuthology.orchestra.run.smithi042.stdout:4062 writesame 0xb4f26b4 thru 0xb5020bd data_size 0xad(0xfa0a bytes) 2024-09-16T13:00:11.344 INFO:teuthology.orchestra.run.smithi042.stdout:4064 read 0x3cd1b47 thru 0x3ce042a (0xe8e4 bytes) 2024-09-16T13:00:11.344 INFO:teuthology.orchestra.run.smithi042.stdout:4065 punch from 0xa4ccecf to 0xa4d9f6d, (0xd09e bytes) 2024-09-16T13:00:11.344 INFO:teuthology.orchestra.run.smithi042.stdout:4067 read 0xa2eaddf thru 0xa2eec86 (0x3ea8 bytes) 2024-09-16T13:00:11.344 INFO:teuthology.orchestra.run.smithi042.stdout:4069 read 0x63e1037 thru 0x63f081a (0xf7e4 bytes) 2024-09-16T13:00:11.344 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:11.344 INFO:teuthology.orchestra.run.smithi042.stdout:4071 read 0x328fe64 thru 0x3299c33 (0x9dd0 bytes) 2024-09-16T13:00:11.344 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:11.344 INFO:teuthology.orchestra.run.smithi042.stdout:4073 punch from 0x5620fdf to 0x5623773, (0x2794 bytes) 2024-09-16T13:00:11.344 INFO:teuthology.orchestra.run.smithi042.stdout:4077 write 0x9f1c7c5 thru 0x9f1e8aa (0x20e6 bytes) 2024-09-16T13:00:11.350 INFO:teuthology.orchestra.run.smithi042.stdout:4078 punch from 0xa625592 to 0xa632c44, (0xd6b2 bytes) 2024-09-16T13:00:11.350 INFO:teuthology.orchestra.run.smithi042.stdout:4079 punch from 0x4c66a96 to 0x4c6a0be, (0x3628 bytes) 2024-09-16T13:00:11.350 INFO:teuthology.orchestra.run.smithi042.stdout:4080 read 0x7c2caae thru 0x7c2f8f7 (0x2e4a bytes) 2024-09-16T13:00:11.350 INFO:teuthology.orchestra.run.smithi042.stdout:4081 write 0x7e8c9e9 thru 0x7e9002f (0x3647 bytes) 2024-09-16T13:00:11.359 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:11.359 INFO:teuthology.orchestra.run.smithi042.stdout:4085 read 0x2ba7497 thru 0x2bab7bc (0x4326 bytes) 2024-09-16T13:00:11.359 INFO:teuthology.orchestra.run.smithi042.stdout:4087 writesame 0x922190f thru 0x92252fb data_size 0x1(0x39ed bytes) 2024-09-16T13:00:11.366 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:11.366 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:11.373 INFO:teuthology.orchestra.run.smithi042.stdout:4088 writesame 0x9b41c1e thru 0x9b49b02 data_size 0x49(0x7ee5 bytes) 2024-09-16T13:00:11.376 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:11.379 INFO:teuthology.orchestra.run.smithi042.stdout:4089 writesame 0x8b219a3 thru 0x8b2d9ae data_size 0x66(0xc00c bytes) 2024-09-16T13:00:11.387 INFO:teuthology.orchestra.run.smithi042.stdout:4091 write 0x6f3d83d thru 0x6f48115 (0xa8d9 bytes) 2024-09-16T13:00:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:4092 punch from 0x992fb06 to 0x993de5e, (0xe358 bytes) 2024-09-16T13:00:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:4093 read 0x3ea1827 thru 0x3ea8a88 (0x7262 bytes) 2024-09-16T13:00:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:4096 write 0x4aa3e97 thru 0x4aaca84 (0x8bee bytes) 2024-09-16T13:00:11.403 INFO:teuthology.orchestra.run.smithi042.stdout:4097 writesame 0x615ce7f thru 0x616a0fa data_size 0x1(0xd27c bytes) 2024-09-16T13:00:11.409 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:11.402+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:11.447 INFO:teuthology.orchestra.run.smithi042.stdout:4100 write 0xe1ce9cc thru 0xe1d3425 (0x4a5a bytes) 2024-09-16T13:00:11.456 INFO:teuthology.orchestra.run.smithi042.stdout:4101 writesame 0x59148bf thru 0x5920e36 data_size 0x8e(0xc578 bytes) 2024-09-16T13:00:11.466 INFO:teuthology.orchestra.run.smithi042.stdout:4102 write 0xe1918a7 thru 0xe198516 (0x6c70 bytes) 2024-09-16T13:00:11.471 INFO:teuthology.orchestra.run.smithi042.stdout:4104 read 0x4d7f5f thru 0x4de21e (0x62c0 bytes) 2024-09-16T13:00:11.472 INFO:teuthology.orchestra.run.smithi042.stdout:4105 read 0x99f33fb thru 0x99f64d4 (0x30da bytes) 2024-09-16T13:00:11.473 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:11.473 INFO:teuthology.orchestra.run.smithi042.stdout:4107 write 0x1812f59 thru 0x181bf9f (0x9047 bytes) 2024-09-16T13:00:11.475 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:11.475 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:11.476 INFO:teuthology.orchestra.run.smithi042.stdout:4108 write 0x3986b9 thru 0x399313 (0xc5b bytes) 2024-09-16T13:00:11.484 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:11.504 INFO:teuthology.orchestra.run.smithi042.stdout:4109 read 0x814dfed thru 0x815db23 (0xfb37 bytes) 2024-09-16T13:00:11.505 INFO:teuthology.orchestra.run.smithi042.stdout:4110 read 0xbb06d16 thru 0xbb0d0a4 (0x638f bytes) 2024-09-16T13:00:11.505 INFO:teuthology.orchestra.run.smithi042.stdout:4112 write 0xa3a5df5 thru 0xa3b0529 (0xa735 bytes) 2024-09-16T13:00:11.510 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:11.503+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:11.511 INFO:teuthology.orchestra.run.smithi042.stdout:4114 read 0x23cb341 thru 0x23cdbd5 (0x2895 bytes) 2024-09-16T13:00:11.511 INFO:teuthology.orchestra.run.smithi042.stdout:4115 write 0x601b931 thru 0x602383d (0x7f0d bytes) 2024-09-16T13:00:11.514 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xee501bc 2024-09-16T13:00:11.516 INFO:teuthology.orchestra.run.smithi042.stdout:4116 trunc from 0xe1d3426 to 0xee501bc 2024-09-16T13:00:11.527 INFO:teuthology.orchestra.run.smithi042.stdout:4117 read 0x36ef9c8 thru 0x36fbd85 (0xc3be bytes) 2024-09-16T13:00:11.528 INFO:teuthology.orchestra.run.smithi042.stdout:4118 flatten 2024-09-16T13:00:11.579 INFO:teuthology.orchestra.run.smithi042.stdout:4119 read 0x3e1f726 thru 0x3e29041 (0x991c bytes) 2024-09-16T13:00:11.579 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:11.579 INFO:teuthology.orchestra.run.smithi042.stdout:4121 write 0xb5a20e8 thru 0xb5a3dc0 (0x1cd9 bytes) 2024-09-16T13:00:11.584 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:11.584 INFO:teuthology.orchestra.run.smithi042.stdout:4125 writesame 0xe53b047 thru 0xe547d2b data_size 0x1(0xcce5 bytes) 2024-09-16T13:00:11.585 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:11.586 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:11.595 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:11.611 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:11.606+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:11.620 INFO:teuthology.orchestra.run.smithi042.stdout:4126 trunc from 0xee501bc to 0xa6bf5b8 2024-09-16T13:00:11.664 INFO:teuthology.orchestra.run.smithi042.stdout:4128 writesame 0xdce1be4 thru 0xdce1c6f data_size 0x7(0x8c bytes) 2024-09-16T13:00:11.664 INFO:teuthology.orchestra.run.smithi042.stdout:4130 clone 23 order 19 su 4096 sc 8 2024-09-16T13:00:11.692 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:11.692 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:11.701 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:11.746 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:11.742+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:11.798 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:11.799 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:11.808 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:11.812 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:11.806+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:11.904 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:11.904 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:11.913 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:11.913 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:11.907+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:12.011 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:12.012 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:12.014 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:12.007+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:12.021 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:12.115 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:12.110+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:12.123 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:12.123 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:12.132 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:12.215 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:12.210+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:12.250 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:12.251 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:12.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:12.316 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:12.307+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:12.359 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:12.359 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:12.369 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:12.417 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:12.407+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:12.467 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T13:00:12.468 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T13:00:12.468 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-16T13:00:12.468 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-16T13:00:12.483 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:12.483 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:12.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:12.533 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-16T13:00:12.534 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:12.523+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:12.618 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:12.614+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:12.626 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-16T13:00:12.637 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:12.638 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:12.651 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:12.719 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:12.711+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:12.763 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:12.763 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:12.776 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-16T13:00:12.777 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:12.821 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:12.811+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:12.882 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:12.882 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:12.895 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:12.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-16T13:00:12.922 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:12.918+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:12.959 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:12.959 INFO:teuthology.orchestra.run.smithi042.stdout:4134 write 0x722aa17 thru 0x72346e3 (0x9ccd bytes) 2024-09-16T13:00:13.025 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:13.018+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:13.058 INFO:teuthology.orchestra.run.smithi042.stdout:4135 read 0x5e78bf0 thru 0x5e7d7fc (0x4c0d bytes) 2024-09-16T13:00:13.060 INFO:teuthology.orchestra.run.smithi042.stdout:4136 read 0xa913d11 thru 0xa91789e (0x3b8e bytes) 2024-09-16T13:00:13.060 INFO:teuthology.orchestra.run.smithi042.stdout:4138 writesame 0xbcf024d thru 0xbcf801b data_size 0x2b(0x7dcf bytes) 2024-09-16T13:00:13.072 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:13.072 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:13.076 INFO:teuthology.orchestra.run.smithi042.stdout:4139 punch from 0x40d0de4 to 0x40dd5e7, (0xc803 bytes) 2024-09-16T13:00:13.077 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:13.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:13.089 INFO:teuthology.orchestra.run.smithi042.stdout:4141 trunc from 0xdce1c70 to 0x5522e6f 2024-09-16T13:00:13.126 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:13.119+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:13.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-16T13:00:13.221 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T13:00:13.224 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:13.224 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:13.229 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:13.219+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:13.238 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:13.247 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:13.285 INFO:teuthology.orchestra.run.smithi042.stdout:4143 write 0x15b193a thru 0x15b6b0d (0x51d4 bytes) 2024-09-16T13:00:13.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-16T13:00:13.345 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:13.346 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:13.346 INFO:teuthology.orchestra.run.smithi042.stdout:4146 write 0x548176 thru 0x54d973 (0x57fe bytes) 2024-09-16T13:00:13.346 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:13.339+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:13.376 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:13.377 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:13.390 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:13.434 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:13.427+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:13.445 INFO:teuthology.orchestra.run.smithi042.stdout:4147 write 0xc743cf3 thru 0xc750542 (0xc850 bytes) 2024-09-16T13:00:13.469 INFO:teuthology.orchestra.run.smithi042.stdout:4148 read 0x927a241 thru 0x9288c24 (0xe9e4 bytes) 2024-09-16T13:00:13.469 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:13.469 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:13.477 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:13.481 INFO:teuthology.orchestra.run.smithi042.stdout:4151 writesame 0xd086ed9 thru 0xd09347f data_size 0x1(0xc5a7 bytes) 2024-09-16T13:00:13.503 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:13.504 INFO:teuthology.orchestra.run.smithi042.stdout:4152 trunc from 0xd093480 to 0x6beba5 2024-09-16T13:00:13.529 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:13.519+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:13.542 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:13.569 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-16T13:00:13.574 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:13.574 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:13.574 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:13.574 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:13.590 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:13.599 INFO:teuthology.orchestra.run.smithi042.stdout:4153 write 0xac31a87 thru 0xac3394c (0x1ec6 bytes) 2024-09-16T13:00:13.601 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:00:13.616 INFO:teuthology.orchestra.run.smithi042.stdout:4155 trunc from 0xac3394d to 0xec757a5 2024-09-16T13:00:13.626 INFO:teuthology.orchestra.run.smithi042.stdout:4156 write 0x926d342 thru 0x927c76f (0xf42e bytes) 2024-09-16T13:00:13.629 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:13.630 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:13.623+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:13.647 INFO:teuthology.orchestra.run.smithi042.stdout:4158 punch from 0xa96d5ff to 0xa96f498, (0x1e99 bytes) 2024-09-16T13:00:13.647 INFO:teuthology.orchestra.run.smithi042.stdout:4159 read 0x8e27b26 thru 0x8e344c0 (0xc99b bytes) 2024-09-16T13:00:13.647 INFO:teuthology.orchestra.run.smithi042.stdout:4160 read 0xc45ba10 thru 0xc4635f1 (0x7be2 bytes) 2024-09-16T13:00:13.647 INFO:teuthology.orchestra.run.smithi042.stdout:4161 write 0x8bca6e3 thru 0x8bcc4f4 (0x1e12 bytes) 2024-09-16T13:00:13.654 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:13.654 INFO:teuthology.orchestra.run.smithi042.stdout:4163 writesame 0x20bb3f thru 0x212717 data_size 0x1(0x6bd9 bytes) 2024-09-16T13:00:13.694 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:13.697 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:13.697 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:13.697 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:13.698 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:13.698 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:00:13.698 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:13.698 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:13.698 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:13.730 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:13.726+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:13.740 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:13.740 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:13.740 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:13.740 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:13.740 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:13.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:13.784 INFO:teuthology.orchestra.run.smithi042.stdout:4164 write 0xcdb40b5 thru 0xcdbc86b (0x87b7 bytes) 2024-09-16T13:00:13.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-16T13:00:13.802 INFO:teuthology.orchestra.run.smithi042.stdout:4165 write 0x82997ba thru 0x829ee40 (0x5687 bytes) 2024-09-16T13:00:13.818 INFO:teuthology.orchestra.run.smithi042.stdout:4166 write 0x25622ad thru 0x25650af (0x2e03 bytes) 2024-09-16T13:00:13.828 INFO:teuthology.orchestra.run.smithi042.stdout:4167 read 0xc98d576 thru 0xc992e0d (0x5898 bytes) 2024-09-16T13:00:13.829 INFO:teuthology.orchestra.run.smithi042.stdout:4168 writesame 0x6d3b30c thru 0x6d4241d data_size 0x29(0x7112 bytes) 2024-09-16T13:00:13.831 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:13.823+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:13.844 INFO:teuthology.orchestra.run.smithi042.stdout:4170 read 0x5678e91 thru 0x5686a8a (0xdbfa bytes) 2024-09-16T13:00:13.844 INFO:teuthology.orchestra.run.smithi042.stdout:4171 write 0xe218712 thru 0xe225be5 (0xd4d4 bytes) 2024-09-16T13:00:13.860 INFO:teuthology.orchestra.run.smithi042.stdout:4172 punch from 0x76d821c to 0x76d9dbd, (0x1ba1 bytes) 2024-09-16T13:00:13.860 INFO:teuthology.orchestra.run.smithi042.stdout:4173 write 0x48aa40d thru 0x48b6bee (0xc7e2 bytes) 2024-09-16T13:00:13.876 INFO:teuthology.orchestra.run.smithi042.stdout:4174 read 0xcab3711 thru 0xcab494b (0x123b bytes) 2024-09-16T13:00:13.883 INFO:teuthology.orchestra.run.smithi042.stdout:4175 trunc from 0xec757a5 to 0xa4bbe04 2024-09-16T13:00:13.896 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47613 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3571712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5344201050000001, 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054671112000000001, 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47613 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5699999999999999e-07 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357529+0000", 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1459999999999999e-06 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357532+0000", 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357534+0000", 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9089999999999998e-06 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412120+0000", 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054585897000000001 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412159+0000", 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8967000000000001e-05 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412199+0000", 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0068999999999999e-05 2024-09-16T13:00:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47614 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3588096~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357536+0000", 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.534411408, 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054770905000000002, 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47614 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357536+0000", 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357536+0000", 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357537+0000", 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357538+0000", 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0950000000000001e-06 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357541+0000", 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9979999999999999e-06 2024-09-16T13:00:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357543+0000", 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9539999999999998e-06 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412227+0000", 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054684501000000003 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412266+0000", 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8764999999999999e-05 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412307+0000", 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1341000000000003e-05 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47615 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3604480~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5344031029999998, 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054886719, 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47615 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.41e-07 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357546+0000", 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.099e-06 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357549+0000", 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8820000000000001e-06 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357551+0000", 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.852e-06 2024-09-16T13:00:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412335+0000", 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054784294999999997 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412392+0000", 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6938000000000003e-05 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412431+0000", 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T13:00:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47616 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3620864~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5343950240000002, 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054998867, 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47616 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4900000000000002e-07 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357554+0000", 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0669999999999999e-06 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357557+0000", 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9079999999999999e-06 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357559+0000", 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9580000000000001e-06 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412469+0000", 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054909873999999997 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412512+0000", 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3430999999999998e-05 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412552+0000", 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9379999999999999e-05 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47617 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3637248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5343868600000001, 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055098616000000003, 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47617 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6e-07 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357562+0000", 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0809999999999999e-06 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357565+0000", 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-16T13:00:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357567+0000", 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412578+0000", 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055010816999999997 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412615+0000", 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7629999999999997e-05 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412660+0000", 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4116999999999998e-05 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47618 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3653632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5343775559999999, 2024-09-16T13:00:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055192228000000003, 2024-09-16T13:00:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47618 2024-09-16T13:00:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3099999999999999e-07 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357573+0000", 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9919999999999999e-06 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357577+0000", 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.208e-06 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357579+0000", 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412685+0000", 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055105981999999998 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412723+0000", 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8028999999999998e-05 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412762+0000", 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9221999999999998e-05 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47619 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3670016~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357581+0000", 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5343664079999999, 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055287718999999999, 2024-09-16T13:00:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47619 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357581+0000", 2024-09-16T13:00:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357581+0000", 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357582+0000", 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6100000000000002e-07 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357583+0000", 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.085e-06 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357586+0000", 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.931e-06 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357588+0000", 2024-09-16T13:00:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412788+0000", 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055200013999999999 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412827+0000", 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9002999999999999e-05 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412869+0000", 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2546000000000002e-05 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47620 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3686400~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357590+0000", 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5343574699999998, 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0553897, 2024-09-16T13:00:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47620 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357590+0000", 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357590+0000", 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357591+0000", 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357592+0000", 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.06e-06 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357595+0000", 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.999e-06 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357597+0000", 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9640000000000002e-06 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412900+0000", 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055303123000000003 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412941+0000", 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0907e-05 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412980+0000", 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47621 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3702784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357598+0000", 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.534349395, 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055490600000000001, 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47621 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357598+0000", 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357598+0000", 2024-09-16T13:00:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357599+0000", 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5199999999999998e-07 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357600+0000", 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357603+0000", 2024-09-16T13:00:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9560000000000002e-06 2024-09-16T13:00:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357606+0000", 2024-09-16T13:00:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1300000000000001e-06 2024-09-16T13:00:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413012+0000", 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055406186000000003 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413051+0000", 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8686999999999997e-05 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413089+0000", 2024-09-16T13:00:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8223000000000003e-05 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47622 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3719168~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5343398420000001, 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055581815, 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47622 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4699999999999998e-07 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357609+0000", 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7399999999999991e-07 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357612+0000", 2024-09-16T13:00:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9799999999999998e-06 2024-09-16T13:00:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357615+0000", 2024-09-16T13:00:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.269e-06 2024-09-16T13:00:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413115+0000", 2024-09-16T13:00:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055499522000000003 2024-09-16T13:00:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413154+0000", 2024-09-16T13:00:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8584999999999997e-05 2024-09-16T13:00:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413190+0000", 2024-09-16T13:00:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6238000000000001e-05 2024-09-16T13:00:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47623 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357617+0000", 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5343303860000002, 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055603054999999998, 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47623 2024-09-16T13:00:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357617+0000", 2024-09-16T13:00:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357617+0000", 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357618+0000", 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6199999999999999e-07 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357619+0000", 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.032e-06 2024-09-16T13:00:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357622+0000", 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9679999999999998e-06 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357624+0000", 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413215+0000", 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055591451 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413220+0000", 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4620000000000004e-06 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47624 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357625+0000", 2024-09-16T13:00:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5343223730000002, 2024-09-16T13:00:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055617605000000001, 2024-09-16T13:00:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47624 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357625+0000", 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357625+0000", 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357626+0000", 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4900000000000002e-07 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357627+0000", 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2139999999999999e-06 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357630+0000", 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9409999999999999e-06 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357633+0000", 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6960000000000001e-06 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413236+0000", 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055603303999999999 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413243+0000", 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2010000000000002e-06 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47625 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5343118150000001, 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055628134000000003, 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47625 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357637+0000", 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5300000000000002e-07 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357640+0000", 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357642+0000", 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9130000000000001e-06 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413257+0000", 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055615255000000002 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413264+0000", 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.849e-06 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47626 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5343038289999997, 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055641412000000001, 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47626 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5600000000000002e-07 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357645+0000", 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-06 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357648+0000", 2024-09-16T13:00:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9859999999999999e-06 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357650+0000", 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.922e-06 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413278+0000", 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055628089999999998 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413285+0000", 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1289999999999998e-06 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47627 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.534295658, 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055653995999999997, 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47627 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357654+0000", 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.212e-06 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357656+0000", 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.897e-06 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357658+0000", 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8920000000000001e-06 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413299+0000", 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055640991000000001 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413306+0000", 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47628 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357660+0000", 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5342874659999999, 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055666457000000003, 2024-09-16T13:00:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47628 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357660+0000", 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357660+0000", 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357661+0000", 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5499999999999999e-07 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357662+0000", 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.065e-06 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357665+0000", 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357668+0000", 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-16T13:00:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413320+0000", 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055652245000000003 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413327+0000", 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7989999999999996e-06 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47629 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357670+0000", 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5342775570000002, 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055676011999999997, 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47629 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357670+0000", 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357670+0000", 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357671+0000", 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4699999999999998e-07 2024-09-16T13:00:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357672+0000", 2024-09-16T13:00:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.015e-06 2024-09-16T13:00:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357675+0000", 2024-09-16T13:00:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0359999999999998e-06 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357677+0000", 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.481e-06 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413340+0000", 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662752000000003 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413346+0000", 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4810000000000003e-06 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47630 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5342687929999999, 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055688423000000001, 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47630 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357680+0000", 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.11e-06 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357683+0000", 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1250000000000001e-06 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357686+0000", 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-06 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413360+0000", 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055674003 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413367+0000", 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9189999999999999e-06 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47631 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357688+0000", 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5342602869999999, 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055700309000000003, 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47631 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357688+0000", 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357688+0000", 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357733+0000", 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5716000000000001e-05 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357740+0000", 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3740000000000003e-06 2024-09-16T13:00:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357748+0000", 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4919999999999993e-06 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357753+0000", 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1259999999999997e-06 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413381+0000", 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055627958999999998 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413388+0000", 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47632 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357759+0000", 2024-09-16T13:00:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.5341888109999999, 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055652716999999997, 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47632 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357759+0000", 2024-09-16T13:00:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:13.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357759+0000", 2024-09-16T13:00:13.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:13.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:13.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359561+0000", 2024-09-16T13:00:13.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018016060000000001 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359564+0000", 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359577+0000", 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3087e-05 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:13.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359582+0000", 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2549999999999999e-06 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413404+0000", 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.053822163999999999 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413412+0000", 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0330000000000008e-06 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:13.987 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:13.988 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:13.989 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:00:13.990 INFO:teuthology.orchestra.run.smithi042.stdout:4176 write 0xb2bc9f6 thru 0xb2c8fc0 (0xc5cb bytes) 2024-09-16T13:00:13.990 INFO:teuthology.orchestra.run.smithi042.stdout:4177 trunc from 0xb2c8fc1 to 0x9ebeb1 2024-09-16T13:00:13.991 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:13.982+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:13.994 INFO:teuthology.orchestra.run.smithi042.stdout:4178 write 0x55d4905 thru 0x55d6fa2 (0x269e bytes) 2024-09-16T13:00:14.004 INFO:teuthology.orchestra.run.smithi042.stdout:4180 read 0x3989f1b thru 0x3992df1 (0x8ed7 bytes) 2024-09-16T13:00:14.005 INFO:teuthology.orchestra.run.smithi042.stdout:4181 read 0x1c4396b thru 0x1c4f23b (0xb8d1 bytes) 2024-09-16T13:00:14.005 INFO:teuthology.orchestra.run.smithi042.stdout:4182 read 0xa53bcf thru 0xa5ca16 (0x8e48 bytes) 2024-09-16T13:00:14.005 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.005 INFO:teuthology.orchestra.run.smithi042.stdout:4185 punch from 0x117eb2d to 0x1184c9c, (0x616f bytes) 2024-09-16T13:00:14.005 INFO:teuthology.orchestra.run.smithi042.stdout:4186 punch from 0x2640f02 to 0x26441b0, (0x32ae bytes) 2024-09-16T13:00:14.023 INFO:teuthology.orchestra.run.smithi042.stdout:4187 write 0x9029c3f thru 0x902b3f6 (0x17b8 bytes) 2024-09-16T13:00:14.033 INFO:teuthology.orchestra.run.smithi042.stdout:4188 read 0x1b9f392 thru 0x1b9fdc8 (0xa37 bytes) 2024-09-16T13:00:14.033 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.049 INFO:teuthology.orchestra.run.smithi042.stdout:4190 write 0xb3348bf thru 0xb33cb89 (0x82cb bytes) 2024-09-16T13:00:14.067 INFO:teuthology.orchestra.run.smithi042.stdout:4194 read 0x8c26c51 thru 0x8c3265c (0xba0c bytes) 2024-09-16T13:00:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:4196 trunc from 0xb33cb8a to 0xce53029 2024-09-16T13:00:14.082 INFO:teuthology.orchestra.run.smithi042.stdout:4197 write 0x2776abf thru 0x2780a80 (0x9fc2 bytes) 2024-09-16T13:00:14.090 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:14.082+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:14.104 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:14.105 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:14.106 INFO:teuthology.orchestra.run.smithi042.stdout:4198 trunc from 0xce53029 to 0xbcce280 2024-09-16T13:00:14.115 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:00:14.116 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.116 INFO:teuthology.orchestra.run.smithi042.stdout:4201 read 0x55c9b12 thru 0x55d01fe (0x66ed bytes) 2024-09-16T13:00:14.137 INFO:teuthology.orchestra.run.smithi042.stdout:4202 trunc from 0xbcce280 to 0x26082d9 2024-09-16T13:00:14.163 INFO:teuthology.orchestra.run.smithi042.stdout:4204 read 0x456657 thru 0x4649cb (0xe375 bytes) 2024-09-16T13:00:14.164 INFO:teuthology.orchestra.run.smithi042.stdout:4206 read 0xc7ca16 thru 0xc8a21f (0xd80a bytes) 2024-09-16T13:00:14.164 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.164 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.164 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.164 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.186 INFO:teuthology.orchestra.run.smithi042.stdout:4213 write 0x759cd45 thru 0x75a798b (0xac47 bytes) 2024-09-16T13:00:14.191 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:14.183+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:14.206 INFO:teuthology.orchestra.run.smithi042.stdout:4216 read 0x5280014 thru 0x5284fb9 (0x4fa6 bytes) 2024-09-16T13:00:14.212 INFO:teuthology.orchestra.run.smithi042.stdout:4217 trunc from 0x75a798c to 0xa73a5d2 2024-09-16T13:00:14.216 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:14.216 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:14.216 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:14.216 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:14.216 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:14.224 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.224 INFO:teuthology.orchestra.run.smithi042.stdout:4220 read 0x7bc8b42 thru 0x7bcafb5 (0x2474 bytes) 2024-09-16T13:00:14.225 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.226 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:00:14.227 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T13:00:14.242 INFO:teuthology.orchestra.run.smithi042.stdout:4222 write 0xbf86354 thru 0xbf8d858 (0x7505 bytes) 2024-09-16T13:00:14.277 INFO:teuthology.orchestra.run.smithi042.stdout:4223 write 0x92a0640 thru 0x92a08d7 (0x298 bytes) 2024-09-16T13:00:14.292 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:14.286+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:14.301 INFO:teuthology.orchestra.run.smithi042.stdout:4224 write 0xe85bd64 thru 0xe86180b (0x5aa8 bytes) 2024-09-16T13:00:14.318 INFO:teuthology.orchestra.run.smithi042.stdout:4225 write 0xbe428e2 thru 0xbe4a55a (0x7c79 bytes) 2024-09-16T13:00:14.322 INFO:teuthology.orchestra.run.smithi042.stdout:4226 write 0x58cac33 thru 0x58d6c11 (0xbfdf bytes) 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:590 4.e 4:774e5493:::rbd_data.11b59d02cc38.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 56.987637114999998, 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034258392999999998, 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 590 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339269+0000", 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339271+0000", 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339281+0000", 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6670000000000008e-06 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339307+0000", 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5907000000000001e-05 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339317+0000", 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0356999999999999e-05 2024-09-16T13:00:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339412+0000", 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4248999999999998e-05 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.339533+0000", 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012184900000000001 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.347489+0000", 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079559599999999998 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373423+0000", 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025933971 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373435+0000", 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1924e-05 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373492+0000", 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6842999999999998e-05 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:17.373528+0000", 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5063 4.13 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22711298~51120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.017492316999999, 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111447293, 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5063 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309414+0000", 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.309416+0000", 2024-09-16T13:00:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5859999999999999e-06 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358861+0000", 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049444717999999999 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358893+0000", 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2279e-05 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.358902+0000", 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4390000000000006e-06 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359062+0000", 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016014100000000001 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.359330+0000", 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026734699999999997 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.362566+0000", 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032358769999999999 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.417541+0000", 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054975345000000002 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420706+0000", 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031651380000000001 2024-09-16T13:00:14.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420790+0000", 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4283999999999995e-05 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420862+0000", 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1138999999999998e-05 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5225 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.522336996999996, 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027003329999999999, 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5225 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.343 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804570+0000", 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804571+0000", 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6309999999999999e-06 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804579+0000", 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0909999999999993e-06 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804600+0000", 2024-09-16T13:00:14.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1114000000000001e-05 2024-09-16T13:00:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804613+0000", 2024-09-16T13:00:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3040000000000001e-05 2024-09-16T13:00:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804713+0000", 2024-09-16T13:00:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9854999999999999e-05 2024-09-16T13:00:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.804844+0000", 2024-09-16T13:00:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130963 2024-09-16T13:00:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.810955+0000", 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061105880000000001 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831495+0000", 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020540467 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831507+0000", 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2024-09-16T13:00:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831544+0000", 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7413000000000002e-05 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.831573+0000", 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8836e-05 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5504 4.e 4:7730d5a1:::rbd_data.11b53fd98b75.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.858633390999998, 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020881448, 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5504 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468273+0000", 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468276+0000", 2024-09-16T13:00:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8370000000000001e-06 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468290+0000", 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.383e-05 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468327+0000", 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468341+0000", 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.331e-05 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468439+0000", 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8746999999999995e-05 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.468593+0000", 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015343999999999999 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.472806+0000", 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042129849999999998 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488823+0000", 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016017686999999999 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.488936+0000", 2024-09-16T13:00:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112074 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489029+0000", 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3801000000000005e-05 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.489155+0000", 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125248 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8354 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.045539321, 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016884742000000001, 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8354 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281367+0000", 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281370+0000", 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3159999999999999e-06 2024-09-16T13:00:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281382+0000", 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281413+0000", 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1519000000000001e-05 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.281435+0000", 2024-09-16T13:00:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1812999999999999e-05 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292521+0000", 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011085915999999999 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.292628+0000", 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107311 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.293108+0000", 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047976600000000001 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.297972+0000", 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048640200000000001 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298174+0000", 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020222 2024-09-16T13:00:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298226+0000", 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2058000000000002e-05 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.298252+0000", 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5743000000000001e-05 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8677 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.316948351000001, 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01642155, 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8677 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009958+0000", 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009959+0000", 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2899999999999999e-07 2024-09-16T13:00:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009961+0000", 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3760000000000002e-06 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009965+0000", 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009968+0000", 2024-09-16T13:00:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1190000000000001e-06 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023570+0000", 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013601317999999999 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.023695+0000", 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012563400000000001 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.024191+0000", 2024-09-16T13:00:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000495434 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026285+0000", 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020948360000000001 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026295+0000", 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8889999999999997e-06 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026338+0000", 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2253999999999999e-05 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.026380+0000", 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2153000000000001e-05 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8678 4.e 4:77b45050:::rbd_object_map.11b55eb65e78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.316934099000001, 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020138508999999999, 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8678 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009972+0000", 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009973+0000", 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3000000000000002e-07 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009975+0000", 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4739999999999999e-06 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009980+0000", 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4399999999999998e-06 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.009983+0000", 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1209999999999998e-06 2024-09-16T13:00:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025464+0000", 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015480918 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025536+0000", 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2212000000000006e-05 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.025768+0000", 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000232186 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029964+0000", 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041957899999999996 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.029981+0000", 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6634e-05 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030070+0000", 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9487999999999994e-05 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.030111+0000", 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0915999999999999e-05 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8718 4.1b 4:dbb4ad27:::rbd_data.11b55eb65e78.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.240619138, 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028260378999999999, 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8718 2024-09-16T13:00:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086287+0000", 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086290+0000", 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1950000000000002e-06 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086300+0000", 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0458999999999999e-05 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086331+0000", 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1117e-05 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086344+0000", 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3191e-05 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086423+0000", 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8288999999999995e-05 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.086530+0000", 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010720799999999999 2024-09-16T13:00:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.091612+0000", 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050817429999999997 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114274+0000", 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022662832000000001 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114386+0000", 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011155899999999999 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114475+0000", 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9043999999999997e-05 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:37.114548+0000", 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2742000000000003e-05 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24294 4.1b 4:d9b96624:::rbd_data.11b59afb69bb.0000000000000000:16 [read 1413120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.856295641999999, 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016422685999999999, 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24294 2024-09-16T13:00:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470611+0000", 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5499999999999999e-07 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470612+0000", 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1790000000000001e-06 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470615+0000", 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0580000000000002e-06 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470618+0000", 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.103e-06 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486878+0000", 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016260398999999998 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486954+0000", 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5763e-05 2024-09-16T13:00:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487034+0000", 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9929e-05 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24328 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1339392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.856119785000001, 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016079547999999999, 2024-09-16T13:00:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24328 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470787+0000", 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0699999999999998e-07 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470789+0000", 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9659999999999999e-06 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470795+0000", 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4110000000000004e-06 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470798+0000", 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5499999999999999e-06 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486710+0000", 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015911536 2024-09-16T13:00:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486790+0000", 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9912000000000003e-05 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486866+0000", 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6766000000000003e-05 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24329 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1368064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.856094547, 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016167826999999999, 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24329 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470812+0000", 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9999999999999998e-07 2024-09-16T13:00:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470815+0000", 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6900000000000001e-06 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470821+0000", 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9050000000000002e-06 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470823+0000", 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486891+0000", 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016068000999999998 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486933+0000", 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1711000000000001e-05 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486980+0000", 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6643e-05 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24330 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.856080937, 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016312199999999999, 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24330 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470826+0000", 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6199999999999999e-07 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470827+0000", 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.333e-06 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470831+0000", 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.348e-06 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470832+0000", 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.849e-06 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.486993+0000", 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016160834999999998 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487060+0000", 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6748000000000005e-05 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487138+0000", 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7824999999999994e-05 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24331 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1425408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.85607224, 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016546297000000001, 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24331 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470834+0000", 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470835+0000", 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4999999999999999e-07 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470836+0000", 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2869999999999999e-06 2024-09-16T13:00:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470839+0000", 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0479999999999999e-06 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.470841+0000", 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0090000000000002e-06 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487212+0000", 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016370609000000001 2024-09-16T13:00:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487288+0000", 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6584999999999999e-05 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487381+0000", 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2509000000000005e-05 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24358 4.1b 4:d9b96624:::rbd_data.11b59afb69bb.0000000000000000:16 [read 1417216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.854891526999999, 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015994596, 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24358 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472015+0000", 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472016+0000", 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5199999999999996e-07 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472017+0000", 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4249999999999999e-06 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472021+0000", 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.596e-06 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472024+0000", 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5860000000000001e-06 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487784+0000", 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015760066999999999 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.487912+0000", 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012761900000000001 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488010+0000", 2024-09-16T13:00:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7651000000000004e-05 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24393 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1372160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.854591793000001, 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016028144000000001, 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24393 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472315+0000", 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9900000000000001e-07 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472317+0000", 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.547e-06 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472321+0000", 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8970000000000001e-06 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472325+0000", 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7759999999999997e-06 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488206+0000", 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015881025 2024-09-16T13:00:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488272+0000", 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5982000000000006e-05 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488343+0000", 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0518000000000005e-05 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24394 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.854578443999999, 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016119201, 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24394 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472328+0000", 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472329+0000", 2024-09-16T13:00:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9500000000000003e-07 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472330+0000", 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6360000000000001e-06 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472334+0000", 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7730000000000001e-06 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472337+0000", 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7420000000000002e-06 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488362+0000", 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016025094 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488405+0000", 2024-09-16T13:00:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3438e-05 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.488447+0000", 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2023000000000001e-05 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:24395 4.13 4:ce497d10:::rbd_data.11b59afb69bb.0000000000000008:16 [read 1429504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.854567303, 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016957005000000001, 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 24395 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472339+0000", 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472340+0000", 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2099999999999997e-07 2024-09-16T13:00:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472341+0000", 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4190000000000001e-06 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472345+0000", 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8139999999999998e-06 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.472348+0000", 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5780000000000001e-06 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489143+0000", 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016795244000000001 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489213+0000", 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0438000000000006e-05 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:59.489296+0000", 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2991000000000006e-05 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:42621 4.e 4:744fcf8a:::rbd_data.11b52dde2e.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:01.415676+0000", 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.911230717, 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034249963000000001, 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 42621 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415676+0000", 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415676+0000", 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415677+0000", 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.564e-06 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415685+0000", 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9060000000000002e-06 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415710+0000", 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.421e-05 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415718+0000", 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7930000000000003e-06 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415779+0000", 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0191e-05 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.415887+0000", 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108936 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.423612+0000", 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077250340000000004 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449730+0000", 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026117745000000001 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449792+0000", 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2277e-05 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449860+0000", 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7794999999999997e-05 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:01.449926+0000", 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5512000000000006e-05 2024-09-16T13:00:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:44349 4.e 4:744fcf8a:::rbd_data.11b52dde2e.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:04.840761+0000", 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4861460110000007, 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020098765000000001, 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44349 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840761+0000", 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840761+0000", 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840764+0000", 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3330000000000001e-06 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840778+0000", 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3923e-05 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840835+0000", 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6994999999999999e-05 2024-09-16T13:00:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840849+0000", 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4544e-05 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.840972+0000", 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122707 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.841125+0000", 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000152867 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.847862+0000", 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006737219 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.860451+0000", 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012589352999999999 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.860582+0000", 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130061 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.860739+0000", 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000157823 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.860859+0000", 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011993999999999999 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:44403 4.e 4:744fcf8a:::rbd_data.11b52dde2e.0000000000000008:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:04.981408+0000", 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3454986009999992, 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036194363, 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44403 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981408+0000", 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981408+0000", 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981410+0000", 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9470000000000002e-06 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981419+0000", 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5200000000000003e-06 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981443+0000", 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3085999999999999e-05 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981469+0000", 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6573000000000001e-05 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981596+0000", 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012663400000000001 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.981740+0000", 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000144036 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:04.990365+0000", 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086247619999999994 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:05.017381+0000", 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027016209999999999 2024-09-16T13:00:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:05.017406+0000", 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5318999999999999e-05 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:05.017530+0000", 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012376600000000001 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:05.017602+0000", 2024-09-16T13:00:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2509999999999995e-05 2024-09-16T13:00:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.438 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:14.440 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:14.441 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:14.434+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:14.441 INFO:teuthology.orchestra.run.smithi042.stdout:4227 trunc from 0xe86180c to 0x6f5e6ac 2024-09-16T13:00:14.441 INFO:teuthology.orchestra.run.smithi042.stdout:4228 writesame 0xd49449c thru 0xd4a3fc9 data_size 0x15(0xfb2e bytes) 2024-09-16T13:00:14.441 INFO:teuthology.orchestra.run.smithi042.stdout:4229 read 0xd24c417 thru 0xd252b84 (0x676e bytes) 2024-09-16T13:00:14.441 INFO:teuthology.orchestra.run.smithi042.stdout:4230 writesame 0xc7a0a4f thru 0xc7af530 data_size 0x23(0xeae2 bytes) 2024-09-16T13:00:14.441 INFO:teuthology.orchestra.run.smithi042.stdout:4232 writesame 0xe9f53de thru 0xe9f691d data_size 0x22(0x1540 bytes) 2024-09-16T13:00:14.441 INFO:teuthology.orchestra.run.smithi042.stdout:4233 punch from 0xa82b0fc to 0xa82efdd, (0x3ee1 bytes) 2024-09-16T13:00:14.441 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.441 INFO:teuthology.orchestra.run.smithi042.stdout:4235 write 0x31434df thru 0x314879c (0x52be bytes) 2024-09-16T13:00:14.450 INFO:teuthology.orchestra.run.smithi042.stdout:4236 read 0x30912f0 thru 0x309f09b (0xddac bytes) 2024-09-16T13:00:14.451 INFO:teuthology.orchestra.run.smithi042.stdout:4237 read 0xdaab54f thru 0xdab6e64 (0xb916 bytes) 2024-09-16T13:00:14.452 INFO:teuthology.orchestra.run.smithi042.stdout:4238 punch from 0x7948a91 to 0x7952f83, (0xa4f2 bytes) 2024-09-16T13:00:14.452 INFO:teuthology.orchestra.run.smithi042.stdout:4239 write 0xc7b59ef thru 0xc7b8e35 (0x3447 bytes) 2024-09-16T13:00:14.457 INFO:teuthology.orchestra.run.smithi042.stdout:4242 write 0x1cb05fc thru 0x1cb69d7 (0x63dc bytes) 2024-09-16T13:00:14.472 INFO:teuthology.orchestra.run.smithi042.stdout:4243 punch from 0x6dbfcdf to 0x6dcd794, (0xdab5 bytes) 2024-09-16T13:00:14.472 INFO:teuthology.orchestra.run.smithi042.stdout:4244 read 0x52953c6 thru 0x52a1fd1 (0xcc0c bytes) 2024-09-16T13:00:14.472 INFO:teuthology.orchestra.run.smithi042.stdout:4245 read 0xaee0b9e thru 0xaeee512 (0xd975 bytes) 2024-09-16T13:00:14.472 INFO:teuthology.orchestra.run.smithi042.stdout:4246 read 0x47ad32a thru 0x47b2079 (0x4d50 bytes) 2024-09-16T13:00:14.472 INFO:teuthology.orchestra.run.smithi042.stdout:4247 writesame 0x6ec03f8 thru 0x6ec2365 data_size 0x1b(0x1f6e bytes) 2024-09-16T13:00:14.479 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.480 INFO:teuthology.orchestra.run.smithi042.stdout:4249 read 0x38a268b thru 0x38aba1b (0x9391 bytes) 2024-09-16T13:00:14.480 INFO:teuthology.orchestra.run.smithi042.stdout:4250 punch from 0x873a8fe to 0x8741d68, (0x746a bytes) 2024-09-16T13:00:14.480 INFO:teuthology.orchestra.run.smithi042.stdout:4251 writesame 0x28272fd thru 0x282edf5 data_size 0x1(0x7af9 bytes) 2024-09-16T13:00:14.501 INFO:teuthology.orchestra.run.smithi042.stdout:4252 write 0xd89a85a thru 0xd8aa2f2 (0xfa99 bytes) 2024-09-16T13:00:14.518 INFO:teuthology.orchestra.run.smithi042.stdout:4254 write 0xa807fe thru 0xa8a223 (0x9a26 bytes) 2024-09-16T13:00:14.535 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.613 INFO:teuthology.orchestra.run.smithi042.stdout:4258 punch from 0xe0cc4f0 to 0xe0dc2da, (0xfdea bytes) 2024-09-16T13:00:14.614 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.615 INFO:teuthology.orchestra.run.smithi042.stdout:4260 trunc from 0xe9f691e to 0x3137f2a 2024-09-16T13:00:14.615 INFO:teuthology.orchestra.run.smithi042.stdout:4262 write 0x7016241 thru 0x701653c (0x2fc bytes) 2024-09-16T13:00:14.615 INFO:teuthology.orchestra.run.smithi042.stdout:4263 trunc from 0x701653d to 0x4aff5c3 2024-09-16T13:00:14.615 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:14.607+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:14.616 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:14.616 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:14.616 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:14.616 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:14.616 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:14.636 INFO:teuthology.orchestra.run.smithi042.stdout:4266 read 0xed056c thru 0xed2d31 (0x27c6 bytes) 2024-09-16T13:00:14.651 INFO:teuthology.orchestra.run.smithi042.stdout:4267 write 0x4fc7307 thru 0x4fd6c32 (0xf92c bytes) 2024-09-16T13:00:14.669 INFO:teuthology.orchestra.run.smithi042.stdout:4268 punch from 0x46c1c1a to 0x46c7c03, (0x5fe9 bytes) 2024-09-16T13:00:14.669 INFO:teuthology.orchestra.run.smithi042.stdout:4270 read 0x3e41671 thru 0x3e4a5bf (0x8f4f bytes) 2024-09-16T13:00:14.674 INFO:teuthology.orchestra.run.smithi042.stdout:4272 trunc from 0x4fd6c33 to 0x73bf02c 2024-09-16T13:00:14.685 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.685 INFO:teuthology.orchestra.run.smithi042.stdout:4274 punch from 0x6ed2866 to 0x6eda97e, (0x8118 bytes) 2024-09-16T13:00:14.708 INFO:teuthology.orchestra.run.smithi042.stdout:4275 writesame 0xc1a44ff thru 0xc1b21ee data_size 0x70(0xdcf0 bytes) 2024-09-16T13:00:14.708 INFO:teuthology.orchestra.run.smithi042.stdout:4276 punch from 0x3a438a5 to 0x3a4837e, (0x4ad9 bytes) 2024-09-16T13:00:14.709 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.709 INFO:teuthology.orchestra.run.smithi042.stdout:4279 read 0x83699b thru 0x8425b3 (0xbc19 bytes) 2024-09-16T13:00:14.710 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.716 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:14.716 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:14.716 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:14.716 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:14.716 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:14.716 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:14.710+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:14.725 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:14.730 INFO:teuthology.orchestra.run.smithi042.stdout:4283 trunc from 0xc1b21ef to 0x29d3b0a 2024-09-16T13:00:14.817 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:14.807+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:14.823 INFO:teuthology.orchestra.run.smithi042.stdout:4284 write 0x8182555 thru 0x818db7c (0xb628 bytes) 2024-09-16T13:00:14.861 INFO:teuthology.orchestra.run.smithi042.stdout:4285 write 0xaa5adf0 thru 0xaa60abd (0x5cce bytes) 2024-09-16T13:00:14.877 INFO:teuthology.orchestra.run.smithi042.stdout:4288 read 0xa36b659 thru 0xa37abf6 (0xf59e bytes) 2024-09-16T13:00:14.877 INFO:teuthology.orchestra.run.smithi042.stdout:4289 write 0x9beba74 thru 0x9bf1f20 (0x64ad bytes) 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8419 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.475151990000001, 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040293592000000003, 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.881 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8419 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402894+0000", 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402907+0000", 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402940+0000", 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3436000000000003e-05 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402953+0000", 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2840000000000001e-05 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403104+0000", 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150712 2024-09-16T13:00:14.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403202+0000", 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8403e-05 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.411015+0000", 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078133079999999997 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:14.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443018+0000", 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032002348999999999 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443075+0000", 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7703e-05 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443141+0000", 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5705999999999997e-05 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443185+0000", 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4152e-05 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.911919632, 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039945895000000002, 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.887 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48384 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966126+0000", 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1110000000000002e-06 2024-09-16T13:00:14.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966134+0000", 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9740000000000002e-06 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966139+0000", 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4270000000000001e-06 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.005829+0000", 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039690007999999999 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.005928+0000", 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9184000000000004e-05 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006070+0000", 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014171100000000001 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.891 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9119000010000002, 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040120731, 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.892 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48385 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.63e-07 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966146+0000", 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.018e-06 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966154+0000", 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9999999999999996e-06 2024-09-16T13:00:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966169+0000", 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5395000000000001e-05 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006118+0000", 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039948637000000002 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006181+0000", 2024-09-16T13:00:14.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3001000000000002e-05 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006264+0000", 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3217e-05 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:14.896 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9118688779999999, 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040378561, 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48386 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:14.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5199999999999996e-07 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966178+0000", 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.373e-06 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966186+0000", 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3259999999999996e-06 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966202+0000", 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6333000000000001e-05 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006311+0000", 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040108447999999998 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006411+0000", 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100141 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006553+0000", 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014218799999999999 2024-09-16T13:00:14.900 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9118353470000002, 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040537542000000003, 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.901 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48387 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966209+0000", 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4300000000000003e-07 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966211+0000", 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.418e-06 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966219+0000", 2024-09-16T13:00:14.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.627e-06 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966223+0000", 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3019999999999996e-06 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006601+0000", 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040377953000000001 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006665+0000", 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3592999999999999e-05 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006746+0000", 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1106000000000006e-05 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.905 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9118157490000001, 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040793276000000003, 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48388 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5700000000000002e-07 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966230+0000", 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0499999999999999e-06 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966246+0000", 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5543e-05 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966250+0000", 2024-09-16T13:00:14.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0840000000000002e-06 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006794+0000", 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040543691999999999 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006901+0000", 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107754 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007021+0000", 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119596 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9117883490000001, 2024-09-16T13:00:14.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040958857000000001, 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48389 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966256+0000", 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8100000000000003e-07 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966258+0000", 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3829999999999999e-06 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966268+0000", 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8379999999999997e-06 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966272+0000", 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9940000000000002e-06 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007069+0000", 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040796897999999998 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007132+0000", 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3084999999999997e-05 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007214+0000", 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2177999999999998e-05 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9117658710000001, 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041219483000000001, 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48390 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4700000000000002e-07 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966280+0000", 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3e-06 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966286+0000", 2024-09-16T13:00:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.57e-06 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966288+0000", 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4439999999999998e-06 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007262+0000", 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040973242 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007351+0000", 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9339999999999995e-05 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007497+0000", 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014614000000000001 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.911744208, 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041394590000000002, 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48391 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966300+0000", 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966301+0000", 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5510000000000001e-06 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966305+0000", 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1849999999999997e-06 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966310+0000", 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.211e-06 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007546+0000", 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041236341000000003 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007610+0000", 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3534999999999994e-05 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007694+0000", 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4407999999999996e-05 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48392 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 794624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9117315540000002, 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041645095, 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48392 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3599999999999999e-07 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966314+0000", 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7689999999999999e-06 2024-09-16T13:00:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966322+0000", 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9340000000000006e-06 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966325+0000", 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5579999999999999e-06 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007742+0000", 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041417127999999998 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007840+0000", 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8199e-05 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007957+0000", 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011717099999999999 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48393 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9117163580000001, 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041849738999999997, 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48393 2024-09-16T13:00:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966328+0000", 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966340+0000", 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2604e-05 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966344+0000", 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1989999999999999e-06 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966348+0000", 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4039999999999999e-06 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008030+0000", 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041682036999999998 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008093+0000", 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3511999999999998e-05 2024-09-16T13:00:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008177+0000", 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3561999999999997e-05 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48394 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9116928990000002, 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042095443000000003, 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48394 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966352+0000", 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.406e-06 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966356+0000", 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966362+0000", 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2299999999999996e-06 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008232+0000", 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041869361000000001 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008320+0000", 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8492000000000006e-05 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008446+0000", 2024-09-16T13:00:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012591800000000001 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48395 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9116773579999999, 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042265693999999999, 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48395 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966367+0000", 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966368+0000", 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9420000000000002e-06 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966372+0000", 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9049999999999999e-06 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966375+0000", 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4329999999999998e-06 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008483+0000", 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042108244000000003 2024-09-16T13:00:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008546+0000", 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2973999999999997e-05 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008632+0000", 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.585e-05 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48396 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9116662710000001, 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042505605000000002, 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48396 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966378+0000", 2024-09-16T13:00:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966379+0000", 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.147e-06 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966382+0000", 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966385+0000", 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.43e-06 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008677+0000", 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042292191 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008765+0000", 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8027000000000003e-05 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008883+0000", 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117817 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48397 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9116563649999998, 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042627777999999998, 2024-09-16T13:00:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48397 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966388+0000", 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966389+0000", 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.144e-06 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966392+0000", 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5939999999999998e-06 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966398+0000", 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3959999999999997e-06 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008904+0000", 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042505730999999998 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008953+0000", 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9786000000000001e-05 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009015+0000", 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1580000000000003e-05 2024-09-16T13:00:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48398 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9116430229999999, 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043115957000000003, 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48398 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2500000000000001e-07 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966402+0000", 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3120000000000001e-06 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966406+0000", 2024-09-16T13:00:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.653e-06 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966408+0000", 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.322e-06 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009095+0000", 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042687260999999997 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009276+0000", 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000180345 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009516+0000", 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024073899999999999 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48399 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.91163229, 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043478389999999999, 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48399 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966412+0000", 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.03e-07 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966413+0000", 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2410000000000001e-06 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966416+0000", 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6330000000000002e-06 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966422+0000", 2024-09-16T13:00:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2379999999999997e-06 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009539+0000", 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043116865999999997 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009588+0000", 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9731999999999998e-05 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009890+0000", 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030137700000000003 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48400 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.91161856, 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043788502, 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48400 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3999999999999997e-07 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966427+0000", 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.765e-06 2024-09-16T13:00:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966434+0000", 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6989999999999997e-06 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966437+0000", 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1109999999999999e-06 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009969+0000", 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043532556 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010101+0000", 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0001317 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010213+0000", 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011233100000000001 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48401 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9116039479999998, 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043965861000000002, 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48401 2024-09-16T13:00:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966442+0000", 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8300000000000001e-06 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966455+0000", 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2927e-05 2024-09-16T13:00:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966459+0000", 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3850000000000004e-06 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010261+0000", 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043801305999999998 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010335+0000", 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4160999999999997e-05 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010405+0000", 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0716000000000005e-05 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48402 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9115794500000001, 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044179172000000003, 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48402 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969465+0000", 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003000689 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969469+0000", 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5739999999999997e-06 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969483+0000", 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3692e-05 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969488+0000", 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6450000000000004e-06 2024-09-16T13:00:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010444+0000", 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040956030999999997 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010542+0000", 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8486000000000005e-05 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010643+0000", 2024-09-16T13:00:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101055 2024-09-16T13:00:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:14.975 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:14.977 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:14.978 INFO:teuthology.orchestra.run.smithi042.stdout:4290 writesame 0x987ad83 thru 0x9889577 data_size 0x11(0xe7f5 bytes) 2024-09-16T13:00:14.978 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.978 INFO:teuthology.orchestra.run.smithi042.stdout:4294 write 0xc4c4f18 thru 0xc4c8e74 (0x3f5d bytes) 2024-09-16T13:00:14.978 INFO:teuthology.orchestra.run.smithi042.stdout:4295 punch from 0x44e2412 to 0x44e5c66, (0x3854 bytes) 2024-09-16T13:00:14.978 INFO:teuthology.orchestra.run.smithi042.stdout:4296 write 0x7975e4 thru 0x79e4d9 (0x6ef6 bytes) 2024-09-16T13:00:14.978 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.978 INFO:teuthology.orchestra.run.smithi042.stdout:4298 punch from 0x48001f3 to 0x480e839, (0xe646 bytes) 2024-09-16T13:00:14.978 INFO:teuthology.orchestra.run.smithi042.stdout:4299 write 0x2ea2ec7 thru 0x2eab7a4 (0x88de bytes) 2024-09-16T13:00:14.978 INFO:teuthology.orchestra.run.smithi042.stdout:4301 read 0x9e1540b thru 0x9e1fc50 (0xa846 bytes) 2024-09-16T13:00:14.979 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:14.979 INFO:teuthology.orchestra.run.smithi042.stdout:4303 write 0x1216469 thru 0x121cae8 (0x6680 bytes) 2024-09-16T13:00:14.979 INFO:teuthology.orchestra.run.smithi042.stdout:4304 trunc from 0xc4c8e75 to 0x64f342e 2024-09-16T13:00:15.026 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:14.967+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:15.026 INFO:teuthology.orchestra.run.smithi042.stdout:4305 writesame 0xdebb7a0 thru 0xdec4e51 data_size 0x1(0x96b2 bytes) 2024-09-16T13:00:15.026 INFO:teuthology.orchestra.run.smithi042.stdout:4306 punch from 0xa932bbf to 0xa93e11e, (0xb55f bytes) 2024-09-16T13:00:15.027 INFO:teuthology.orchestra.run.smithi042.stdout:4308 writesame 0x13a32f5 thru 0x13ade32 data_size 0x17(0xab3e bytes) 2024-09-16T13:00:15.034 INFO:teuthology.orchestra.run.smithi042.stdout:4309 read 0x3072d56 thru 0x307c8e6 (0x9b91 bytes) 2024-09-16T13:00:15.034 INFO:teuthology.orchestra.run.smithi042.stdout:4312 write 0xb87dd67 thru 0xb88dc8f (0xff29 bytes) 2024-09-16T13:00:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:4315 write 0x7e5ca9a thru 0x7e67ff1 (0xb558 bytes) 2024-09-16T13:00:15.067 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:15.076 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:15.070+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:15.079 INFO:teuthology.orchestra.run.smithi042.stdout:4320 write 0xeb8dbb3 thru 0xeb9ca48 (0xee96 bytes) 2024-09-16T13:00:15.087 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55947 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.078548+0000", 2024-09-16T13:00:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0069463459999999999, 2024-09-16T13:00:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069810849999999997, 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for sub ops", 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55947 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078548+0000", 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078548+0000", 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078548+0000", 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5499999999999998e-07 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078551+0000", 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1549999999999999e-06 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078559+0000", 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5480000000000004e-06 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078566+0000", 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6220000000000003e-06 2024-09-16T13:00:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.081734+0000", 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00316884 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.081899+0000", 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016428100000000001 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.082500+0000", 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060134700000000001 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55948 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.078570+0000", 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.006923813, 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070553079999999997, 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for sub ops", 2024-09-16T13:00:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55948 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078570+0000", 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078570+0000", 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078571+0000", 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2200000000000004e-07 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078574+0000", 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2780000000000002e-06 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078580+0000", 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.835e-06 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078584+0000", 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.109e-06 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.083733+0000", 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051494230000000002 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.083856+0000", 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012217 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.084330+0000", 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047387900000000002 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55949 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.078588+0000", 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0069060659999999998, 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071279890000000004, 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for sub ops", 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55949 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078588+0000", 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078588+0000", 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078589+0000", 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7700000000000004e-07 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078591+0000", 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8229999999999999e-06 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078597+0000", 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078601+0000", 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8260000000000003e-06 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.085317+0000", 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067160939999999997 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.085410+0000", 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3165000000000007e-05 2024-09-16T13:00:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.085693+0000", 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028307400000000002 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55950 4.a 4.de3123aa (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.078606+0000", 2024-09-16T13:00:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0068882049999999997, 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071949559999999997, 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55950 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078606+0000", 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078606+0000", 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078606+0000", 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2399999999999998e-07 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078610+0000", 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1760000000000002e-06 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078615+0000", 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5890000000000002e-06 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078619+0000", 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6090000000000001e-06 2024-09-16T13:00:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55951 4.a 4.de3123aa (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.078631+0000", 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0068628980000000001, 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072273060000000002, 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.105 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55951 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078631+0000", 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078631+0000", 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078632+0000", 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078635+0000", 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9699999999999999e-06 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078641+0000", 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1689999999999997e-06 2024-09-16T13:00:15.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078644+0000", 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6239999999999999e-06 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55952 4.a 4.de3123aa (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:15.108 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0068445190000000003, 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072649480000000002, 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55952 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0800000000000005e-07 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078653+0000", 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7499999999999999e-06 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078663+0000", 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0455000000000001e-05 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078667+0000", 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8770000000000003e-06 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.111 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:55947 4.a e80/60)", 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.084397+0000", 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0010971959999999999, 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015724739999999999, 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.084397+0000", 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.084397+0000", 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.084398+0000", 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5799999999999998e-07 2024-09-16T13:00:15.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.084403+0000", 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2950000000000001e-06 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.084413+0000", 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6520000000000001e-06 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.084420+0000", 2024-09-16T13:00:15.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.939e-06 2024-09-16T13:00:15.115 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.115 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.115 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.115 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.115 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-16T13:00:15.115 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 7 2024-09-16T13:00:15.115 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:15.116 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:15.116 INFO:teuthology.orchestra.run.smithi042.stdout:4322 writesame 0x40b25eb thru 0x40be640 data_size 0x7(0xc056 bytes) 2024-09-16T13:00:15.116 INFO:teuthology.orchestra.run.smithi042.stdout:4323 writesame 0xbd1e237 thru 0xbd2992e data_size 0x14(0xb6f8 bytes) 2024-09-16T13:00:15.117 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:15.129 INFO:teuthology.orchestra.run.smithi042.stdout:4324 write 0x7b25461 thru 0x7b33ee7 (0xea87 bytes) 2024-09-16T13:00:15.146 INFO:teuthology.orchestra.run.smithi042.stdout:4325 write 0x9d1ebe thru 0x9d3ea9 (0x1fec bytes) 2024-09-16T13:00:15.149 INFO:teuthology.orchestra.run.smithi042.stdout:4328 read 0x7fd91e0 thru 0x7fe89e6 (0xf807 bytes) 2024-09-16T13:00:15.150 INFO:teuthology.orchestra.run.smithi042.stdout:4330 punch from 0xb4af405 to 0xb4bb1c1, (0xbdbc bytes) 2024-09-16T13:00:15.150 INFO:teuthology.orchestra.run.smithi042.stdout:4331 read 0xe82d3bb thru 0xe83028b (0x2ed1 bytes) 2024-09-16T13:00:15.151 INFO:teuthology.orchestra.run.smithi042.stdout:4332 write 0x8f64886 thru 0x8f71f9e (0xd719 bytes) 2024-09-16T13:00:15.169 INFO:teuthology.orchestra.run.smithi042.stdout:4333 read 0x2dd03d5 thru 0x2dd3c2b (0x3857 bytes) 2024-09-16T13:00:15.170 INFO:teuthology.orchestra.run.smithi042.stdout:4334 read 0xd28f333 thru 0xd29b5bc (0xc28a bytes) 2024-09-16T13:00:15.177 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:15.171+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:15.179 INFO:teuthology.orchestra.run.smithi042.stdout:4335 write 0xec709f1 thru 0xec7877d (0x7d8d bytes) 2024-09-16T13:00:15.197 INFO:teuthology.orchestra.run.smithi042.stdout:4337 trunc from 0xec7877e to 0xe47320d 2024-09-16T13:00:15.240 INFO:teuthology.orchestra.run.smithi042.stdout:4338 trunc from 0xe47320d to 0x4158ea1 2024-09-16T13:00:15.267 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:15.267 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:15.267 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:15.267 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:15.267 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:15.269 INFO:teuthology.orchestra.run.smithi042.stdout:4339 read 0x1bf122 thru 0x1cce93 (0xdd72 bytes) 2024-09-16T13:00:15.276 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:15.278 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:15.271+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:15.285 INFO:teuthology.orchestra.run.smithi042.stdout:4340 write 0x6f046e6 thru 0x6f060c5 (0x19e0 bytes) 2024-09-16T13:00:15.291 INFO:teuthology.orchestra.run.smithi042.stdout:4341 writesame 0x2edddc6 thru 0x2ee7a6f data_size 0xb(0x9caa bytes) 2024-09-16T13:00:15.297 INFO:teuthology.orchestra.run.smithi042.stdout:4342 writesame 0x53b6e9f thru 0x53bd6ef data_size 0x31(0x6851 bytes) 2024-09-16T13:00:15.314 INFO:teuthology.orchestra.run.smithi042.stdout:4343 writesame 0x2f73b2f thru 0x2f825db data_size 0x1(0xeaad bytes) 2024-09-16T13:00:15.348 INFO:teuthology.orchestra.run.smithi042.stdout:4345 write 0x9b8e984 thru 0x9b953b1 (0x6a2e bytes) 2024-09-16T13:00:15.364 INFO:teuthology.orchestra.run.smithi042.stdout:4346 read 0x22b9475 thru 0x22bf8bd (0x6449 bytes) 2024-09-16T13:00:15.364 INFO:teuthology.orchestra.run.smithi042.stdout:4347 write 0x7b9cbb4 thru 0x7b9d228 (0x675 bytes) 2024-09-16T13:00:15.379 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:15.371+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:15.387 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:15.387 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:15.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:15.387 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:15.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.387 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T13:00:15.388 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:15.388 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.019368311999997, 2024-09-16T13:00:15.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T13:00:15.388 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.388 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.389 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:15.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T13:00:15.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T13:00:15.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T13:00:15.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T13:00:15.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.393 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5076 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.451911+0000", 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.931650640999997, 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020443647999999998, 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.394 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5076 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451913+0000", 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451920+0000", 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5459999999999997e-06 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451938+0000", 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7581999999999999e-05 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451950+0000", 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2033e-05 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452042+0000", 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2533000000000003e-05 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452251+0000", 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020887 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.456761+0000", 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045099959999999996 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472274+0000", 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015512882 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472281+0000", 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3939999999999998e-06 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472324+0000", 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2389999999999999e-05 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472355+0000", 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1436000000000002e-05 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5580 4.12 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.690527582999998, 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034176870999999998, 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5580 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693037+0000", 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0650000000000001e-06 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693045+0000", 2024-09-16T13:00:15.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T13:00:15.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693070+0000", 2024-09-16T13:00:15.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5341999999999999e-05 2024-09-16T13:00:15.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693081+0000", 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0611000000000001e-05 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693182+0000", 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100437 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693309+0000", 2024-09-16T13:00:15.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012741400000000001 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.703090+0000", 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097814800000000004 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726908+0000", 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023817680000000001 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727023+0000", 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011449400000000001 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727130+0000", 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010704099999999999 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727211+0000", 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1675000000000007e-05 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.405 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:18031 4.4 4:2649aabf:::rbd_data.11b51a3eb68b.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~5304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.223045078999998, 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94984657100000003, 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 18031 2024-09-16T13:00:15.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160519+0000", 2024-09-16T13:00:15.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3029999999999998e-06 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160536+0000", 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7181999999999999e-05 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160564+0000", 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971e-05 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160576+0000", 2024-09-16T13:00:15.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160669+0000", 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3282999999999994e-05 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160680+0000", 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107188+0000", 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94650794999999999 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107644+0000", 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00045604499999999997 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.108257+0000", 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061324700000000003 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.109811+0000", 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015538749999999999 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110175+0000", 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036419399999999998 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110285+0000", 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010938900000000001 2024-09-16T13:00:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110364+0000", 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8689000000000005e-05 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:54977 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:13.783737+0000", 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.5998250519999999, 2024-09-16T13:00:15.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014990309, 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 54977 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783737+0000", 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783737+0000", 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783738+0000", 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3900000000000003e-07 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783744+0000", 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4999999999999996e-06 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783757+0000", 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2741999999999999e-05 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783782+0000", 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4904000000000001e-05 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.794567+0000", 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010785021000000001 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.794733+0000", 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000165916 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.795131+0000", 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039803000000000001 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.798244+0000", 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031124780000000001 2024-09-16T13:00:15.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.798530+0000", 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028618200000000001 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.798668+0000", 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013795499999999999 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.798727+0000", 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9642000000000003e-05 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:54978 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:13.783803+0000", 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.5997587339999999, 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015186682999999999, 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.418 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 54978 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783803+0000", 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783803+0000", 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783804+0000", 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2699999999999998e-07 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783811+0000", 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0729999999999999e-06 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783826+0000", 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5139e-05 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.783840+0000", 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.342e-05 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.796531+0000", 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012690882000000001 2024-09-16T13:00:15.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.796678+0000", 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014726100000000001 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.797326+0000", 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000647516 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.798817+0000", 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014917559999999999 2024-09-16T13:00:15.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.798884+0000", 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7170999999999998e-05 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.798944+0000", 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9953e-05 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:13.798990+0000", 2024-09-16T13:00:15.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5584999999999998e-05 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55237 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.082567+0000", 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.300995001, 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017090014000000001, 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.424 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55237 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082567+0000", 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082567+0000", 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082568+0000", 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5599999999999995e-07 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082571+0000", 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.948e-06 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082576+0000", 2024-09-16T13:00:15.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7180000000000001e-06 2024-09-16T13:00:15.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082580+0000", 2024-09-16T13:00:15.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.597e-06 2024-09-16T13:00:15.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.095399+0000", 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012819179 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.095550+0000", 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150631 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.096161+0000", 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061149499999999999 2024-09-16T13:00:15.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.097556+0000", 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013950779999999999 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.099583+0000", 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002026637 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.099626+0000", 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3231e-05 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.099657+0000", 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0944000000000001e-05 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.430 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55315 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.185928+0000", 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1976337770000001, 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017209391000000001, 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55315 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185928+0000", 2024-09-16T13:00:15.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185928+0000", 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185929+0000", 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0109999999999999e-06 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185936+0000", 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8369999999999999e-06 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185961+0000", 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4990999999999998e-05 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185971+0000", 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5400000000000001e-06 2024-09-16T13:00:15.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.197521+0000", 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011550685999999999 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.197655+0000", 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134041 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.198211+0000", 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000555238 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.200496+0000", 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022851640000000001 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.203024+0000", 2024-09-16T13:00:15.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025280849999999998 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.203091+0000", 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7135e-05 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.203138+0000", 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6662999999999997e-05 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.436 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55348 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.242380+0000", 2024-09-16T13:00:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.141181604, 2024-09-16T13:00:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016534284, 2024-09-16T13:00:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.437 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55348 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242380+0000", 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242380+0000", 2024-09-16T13:00:15.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242381+0000", 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242383+0000", 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2879999999999999e-06 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242388+0000", 2024-09-16T13:00:15.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.498e-06 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242391+0000", 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9239999999999999e-06 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.253143+0000", 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010752526 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.253264+0000", 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012047100000000001 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.253836+0000", 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057289099999999996 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.255835+0000", 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019988060000000001 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258786+0000", 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029510439999999999 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258864+0000", 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7324999999999996e-05 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258915+0000", 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1093e-05 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55349 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.242396+0000", 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.141165682, 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016602670999999999, 2024-09-16T13:00:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55349 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242396+0000", 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242396+0000", 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242397+0000", 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.58e-07 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242399+0000", 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3159999999999999e-06 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242404+0000", 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7859999999999996e-06 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242407+0000", 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8169999999999999e-06 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.254710+0000", 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01230288 2024-09-16T13:00:15.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.254807+0000", 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7310000000000002e-05 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.255183+0000", 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037571600000000001 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.257088+0000", 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00190563 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258929+0000", 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018412140000000001 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258963+0000", 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3385e-05 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258999+0000", 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6158999999999997e-05 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55457 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.404534+0000", 2024-09-16T13:00:15.449 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.979027804, 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015424640999999999, 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55457 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404534+0000", 2024-09-16T13:00:15.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404534+0000", 2024-09-16T13:00:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404535+0000", 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-07 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404538+0000", 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3060000000000002e-06 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404545+0000", 2024-09-16T13:00:15.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1770000000000001e-06 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404553+0000", 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6199999999999999e-06 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.416061+0000", 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011507669999999999 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.416176+0000", 2024-09-16T13:00:15.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011506300000000001 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.416553+0000", 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037766600000000003 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.418985+0000", 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024314409999999999 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.419891+0000", 2024-09-16T13:00:15.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00090627199999999996 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.419932+0000", 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0552000000000001e-05 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.419959+0000", 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.722e-05 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55458 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.404581+0000", 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.97898148200000001, 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015441407000000001, 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55458 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404581+0000", 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404581+0000", 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404581+0000", 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9400000000000004e-07 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404586+0000", 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.228e-06 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404594+0000", 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2430000000000005e-06 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.404618+0000", 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3765999999999999e-05 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.417109+0000", 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012491402 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.417161+0000", 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1604000000000003e-05 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.417365+0000", 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020389599999999999 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.419405+0000", 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020409080000000001 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.419967+0000", 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000561835 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.419997+0000", 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9859e-05 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.420022+0000", 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4771999999999999e-05 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55524 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.479697+0000", 2024-09-16T13:00:15.461 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.903865217, 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015131034999999999, 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55524 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479697+0000", 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479697+0000", 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479697+0000", 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7599999999999998e-07 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479699+0000", 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0480000000000001e-06 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479704+0000", 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8860000000000003e-06 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479707+0000", 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6900000000000001e-06 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.489965+0000", 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010257896000000001 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.490205+0000", 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024001099999999999 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.491257+0000", 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010526089999999999 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.493363+0000", 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021055090000000002 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.494633+0000", 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012703370000000001 2024-09-16T13:00:15.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.494746+0000", 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112755 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.494828+0000", 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2018e-05 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55525 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.479709+0000", 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.90385321600000001, 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017363224999999999, 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55525 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.468 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479709+0000", 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479709+0000", 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479709+0000", 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5600000000000002e-07 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479711+0000", 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7540000000000001e-06 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479714+0000", 2024-09-16T13:00:15.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.298e-06 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479718+0000", 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3529999999999997e-06 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.492361+0000", 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012642054 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.492421+0000", 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9993e-05 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.492684+0000", 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026362299999999999 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.494910+0000", 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022256789999999999 2024-09-16T13:00:15.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.496953+0000", 2024-09-16T13:00:15.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020428600000000001 2024-09-16T13:00:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.497021+0000", 2024-09-16T13:00:15.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8291000000000001e-05 2024-09-16T13:00:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.497072+0000", 2024-09-16T13:00:15.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1063999999999998e-05 2024-09-16T13:00:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55699 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.64260347900000003, 2024-09-16T13:00:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036277113999999999, 2024-09-16T13:00:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.479 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55699 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:15.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740960+0000", 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.48e-06 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740967+0000", 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8499999999999996e-06 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740984+0000", 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7555999999999999e-05 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740991+0000", 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6969999999999996e-06 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741042+0000", 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0927000000000002e-05 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741116+0000", 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4016000000000004e-05 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741422+0000", 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030587600000000001 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.742556+0000", 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011342139999999999 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.776943+0000", 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034386727999999998 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.777160+0000", 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021688499999999999 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.777236+0000", 2024-09-16T13:00:15.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5884999999999996e-05 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55733 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.822734+0000", 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.56082835799999997, 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016635594, 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.486 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55733 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822734+0000", 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822734+0000", 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822734+0000", 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3500000000000002e-07 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822737+0000", 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1329999999999999e-06 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822743+0000", 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9259999999999997e-06 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822747+0000", 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.794e-06 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.834106+0000", 2024-09-16T13:00:15.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011359088999999999 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.834191+0000", 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5131999999999997e-05 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.834584+0000", 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039327299999999997 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.837181+0000", 2024-09-16T13:00:15.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002596526 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839315+0000", 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021340320000000001 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839347+0000", 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2271000000000003e-05 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839369+0000", 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1982999999999999e-05 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55734 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.822751+0000", 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.56081122500000002, 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016775394999999999, 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.492 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55734 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822751+0000", 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822751+0000", 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822751+0000", 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.01e-07 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822754+0000", 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6230000000000001e-06 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822760+0000", 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822763+0000", 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.765e-06 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.835261+0000", 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012497291000000001 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.835390+0000", 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012971 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.835924+0000", 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053354900000000002 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.837316+0000", 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013921669999999999 2024-09-16T13:00:15.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839423+0000", 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002106424 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839482+0000", 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9116000000000001e-05 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839526+0000", 2024-09-16T13:00:15.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4468000000000002e-05 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55910 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.034703+0000", 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.348858744, 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015885804, 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55910 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034703+0000", 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034703+0000", 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034704+0000", 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7999999999999995e-07 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034711+0000", 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4019999999999996e-06 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034718+0000", 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0940000000000002e-06 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034723+0000", 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.046578+0000", 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011855286 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.046683+0000", 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104244 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.047047+0000", 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036465800000000002 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.050476+0000", 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034288449999999998 2024-09-16T13:00:15.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.050512+0000", 2024-09-16T13:00:15.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6032999999999999e-05 2024-09-16T13:00:15.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.050553+0000", 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1226000000000001e-05 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.050589+0000", 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5790000000000001e-05 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55911 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.504 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.034728+0000", 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.34883437099999998, 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016693761000000001, 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55911 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034728+0000", 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034728+0000", 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034728+0000", 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7800000000000001e-07 2024-09-16T13:00:15.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034732+0000", 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1769999999999998e-06 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034738+0000", 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034743+0000", 2024-09-16T13:00:15.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4800000000000003e-06 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.048353+0000", 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013609804 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.048449+0000", 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6061000000000001e-05 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.048844+0000", 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039503500000000002 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.050605+0000", 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017609360000000001 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.051361+0000", 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00075586999999999996 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.051391+0000", 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0814e-05 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.051421+0000", 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0026999999999999e-05 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55952 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.30491252600000002, 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016269499, 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55952 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.511 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0800000000000005e-07 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078653+0000", 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7499999999999999e-06 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078663+0000", 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0455000000000001e-05 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078667+0000", 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8770000000000003e-06 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.091010+0000", 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012342674999999999 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.091157+0000", 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014694900000000001 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.091652+0000", 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049516700000000002 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.094248+0000", 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025961970000000002 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.094801+0000", 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055320100000000002 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.094843+0000", 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1353000000000002e-05 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.094919+0000", 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6366999999999996e-05 2024-09-16T13:00:15.516 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.517 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.517 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.517 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:15.519 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:15.519 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:15.514+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:15.520 INFO:teuthology.orchestra.run.smithi042.stdout:4348 write 0xe8c12d2 thru 0xe8c6f51 (0x5c80 bytes) 2024-09-16T13:00:15.520 INFO:teuthology.orchestra.run.smithi042.stdout:4349 read 0xa58ab00 thru 0xa59690e (0xbe0f bytes) 2024-09-16T13:00:15.520 INFO:teuthology.orchestra.run.smithi042.stdout:4351 write 0x9f6c3ea thru 0x9f748dd (0x84f4 bytes) 2024-09-16T13:00:15.520 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:15.520 INFO:teuthology.orchestra.run.smithi042.stdout:4353 trunc from 0xe8c6f52 to 0x1abd3de 2024-09-16T13:00:15.520 INFO:teuthology.orchestra.run.smithi042.stdout:4354 read 0x724069 thru 0x73035f (0xc2f7 bytes) 2024-09-16T13:00:15.520 INFO:teuthology.orchestra.run.smithi042.stdout:4355 writesame 0xa3b9d30 thru 0xa3c2f16 data_size 0x1(0x91e7 bytes) 2024-09-16T13:00:15.520 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:15.520 INFO:teuthology.orchestra.run.smithi042.stdout:4360 writesame 0x121de9f thru 0x122b901 data_size 0x1(0xda63 bytes) 2024-09-16T13:00:15.531 INFO:teuthology.orchestra.run.smithi042.stdout:4361 writesame 0x3b9e7e2 thru 0x3b9f999 data_size 0xe(0x11b8 bytes) 2024-09-16T13:00:15.540 INFO:teuthology.orchestra.run.smithi042.stdout:4362 read 0x618e577 thru 0x6192ba7 (0x4631 bytes) 2024-09-16T13:00:15.540 INFO:teuthology.orchestra.run.smithi042.stdout:4363 read 0x8b91d46 thru 0x8b9fddb (0xe096 bytes) 2024-09-16T13:00:15.540 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:15.540 INFO:teuthology.orchestra.run.smithi042.stdout:4367 write 0x1a9d7ee thru 0x1a9f2d4 (0x1ae7 bytes) 2024-09-16T13:00:15.565 INFO:teuthology.orchestra.run.smithi042.stdout:4368 write 0xd892c14 thru 0xd89acc0 (0x80ad bytes) 2024-09-16T13:00:15.602 INFO:teuthology.orchestra.run.smithi042.stdout:4369 write 0xe65bd07 thru 0xe65e63a (0x2934 bytes) 2024-09-16T13:00:15.616 INFO:teuthology.orchestra.run.smithi042.stdout:4371 write 0x57d1017 thru 0x57dc78d (0xb777 bytes) 2024-09-16T13:00:15.619 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:15.611+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:15.634 INFO:teuthology.orchestra.run.smithi042.stdout:4373 write 0xc592996 thru 0xc5934af (0xb1a bytes) 2024-09-16T13:00:15.635 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:15.635 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:15.635 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:15.635 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:15.640 INFO:teuthology.orchestra.run.smithi042.stdout:4374 writesame 0xc2b3f1f thru 0xc2baac5 data_size 0x1f(0x6ba7 bytes) 2024-09-16T13:00:15.645 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:15.664 INFO:teuthology.orchestra.run.smithi042.stdout:4375 read 0x549bdb9 thru 0x54a1787 (0x59cf bytes) 2024-09-16T13:00:15.665 INFO:teuthology.orchestra.run.smithi042.stdout:4377 write 0x8734864 thru 0x873c63b (0x7dd8 bytes) 2024-09-16T13:00:15.684 INFO:teuthology.orchestra.run.smithi042.stdout:4378 trunc from 0xe65e63b to 0xcdd86c5 2024-09-16T13:00:15.713 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:15.718 INFO:teuthology.orchestra.run.smithi042.stdout:4379 writesame 0xe513628 thru 0xe51615f data_size 0x6(0x2b38 bytes) 2024-09-16T13:00:15.718 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:15.718 INFO:teuthology.orchestra.run.smithi042.stdout:4381 read 0x2ce22ad thru 0x2ce4578 (0x22cc bytes) 2024-09-16T13:00:15.718 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:15.719 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:15.714+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:15.725 INFO:teuthology.orchestra.run.smithi042.stdout:4384 trunc from 0xe516160 to 0xc260508 2024-09-16T13:00:15.743 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:15.744 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:15.748 INFO:teuthology.orchestra.run.smithi042.stdout:4385 write 0x9d549fe thru 0x9d5f6d4 (0xacd7 bytes) 2024-09-16T13:00:15.753 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:15.764 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:15.764 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:15.764 INFO:teuthology.orchestra.run.smithi042.stdout:4388 read 0x31dd497 thru 0x31e6772 (0x92dc bytes) 2024-09-16T13:00:15.764 INFO:teuthology.orchestra.run.smithi042.stdout:4389 writesame 0xad80d31 thru 0xad822ec data_size 0x4(0x15bc bytes) 2024-09-16T13:00:15.771 INFO:teuthology.orchestra.run.smithi042.stdout:4390 punch from 0x26c815b to 0x26c83bb, (0x260 bytes) 2024-09-16T13:00:15.771 INFO:teuthology.orchestra.run.smithi042.stdout:4391 clone 24 order 21 su 8192 sc 10 2024-09-16T13:00:15.820 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:15.811+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:15.850 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:15.850 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:15.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:15.850 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:15.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.850 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 90.295305317, 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:15.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:00:15.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:00:15.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:00:15.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:00:15.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374548+0000", 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.470478732999993, 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010809278, 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 44 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374548+0000", 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374549+0000", 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.037e-06 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374711+0000", 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016226599999999999 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374734+0000", 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2895999999999999e-05 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374746+0000", 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.221e-05 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.380942+0000", 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061959390000000001 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381063+0000", 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121274 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.381255+0000", 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019110500000000001 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384487+0000", 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032327979999999998 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385296+0000", 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080888499999999996 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385332+0000", 2024-09-16T13:00:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5774e-05 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385357+0000", 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5094000000000001e-05 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.470267840999995, 2024-09-16T13:00:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010976025, 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 45 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374760+0000", 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.13e-06 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374769+0000", 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5699999999999999e-06 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374819+0000", 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9882000000000001e-05 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374828+0000", 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-16T13:00:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382329+0000", 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075009009999999999 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382403+0000", 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4208999999999994e-05 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382514+0000", 2024-09-16T13:00:15.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111276 2024-09-16T13:00:15.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385533+0000", 2024-09-16T13:00:15.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030190080000000001 2024-09-16T13:00:15.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385564+0000", 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0926000000000002e-05 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385666+0000", 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101791 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385735+0000", 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8541999999999995e-05 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.470188269999994, 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:00:15.871 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.872 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.872 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:00:15.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.872 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:15.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:15.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:00:15.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:00:15.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:00:15.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:00:15.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.470141014000006, 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.878 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:15.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:00:15.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:00:15.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:00:15.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:00:15.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:00:15.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:00:15.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:00:15.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:00:15.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:00:15.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:00:15.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:00:15.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:00:15.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:00:15.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:00:15.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:00:15.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:00:15.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:15.885 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.469938077999998, 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:00:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:00:15.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.469906816000005, 2024-09-16T13:00:15.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:00:15.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:00:15.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.469871053999995, 2024-09-16T13:00:15.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:00:15.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:00:15.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:00:15.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:00:15.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:00:15.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:00:15.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.469676423999999, 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.903 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:15.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:00:15.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:00:15.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.469629578999999, 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919457+0000", 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 88.925569538000005, 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010623719, 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 69 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919457+0000", 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919458+0000", 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.474e-06 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919621+0000", 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016234800000000001 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919641+0000", 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0336999999999999e-05 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919656+0000", 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4554e-05 2024-09-16T13:00:15.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924273+0000", 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046169610000000002 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924411+0000", 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138286 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.924627+0000", 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021597399999999999 2024-09-16T13:00:15.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928794+0000", 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004167268 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929983+0000", 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011884409999999999 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930039+0000", 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6363000000000002e-05 2024-09-16T13:00:15.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930081+0000", 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1712999999999999e-05 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 88.925353408000007, 2024-09-16T13:00:15.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:00:15.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:00:15.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 88.925299500999998, 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.926 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:00:15.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:00:15.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:00:15.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:00:15.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:00:15.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 88.925257196000004, 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:00:15.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:15.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 88.925098770999995, 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:00:15.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:00:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:15.944 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 88.925049290000004, 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:00:15.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:00:15.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:00:15.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:00:15.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:00:15.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.950 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 88.924918719999994, 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:00:15.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.952 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:15.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:15.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:00:15.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:00:15.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:00:15.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:15.957 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 88.924737207000007, 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:00:15.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:00:15.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.963 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.963 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:15.963 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:15.963 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 88.114209540000004, 2024-09-16T13:00:15.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:00:15.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:00:15.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:00:15.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:00:15.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:00:15.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:00:15.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.969 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.970 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:15.970 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:15.970 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 88.114060585999994, 2024-09-16T13:00:15.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:00:15.970 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:15.970 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:15.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:00:15.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:15.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:00:15.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:15.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:15.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:15.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:00:15.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:00:15.976 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.976 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.976 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.976 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:15.976 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:15.976 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:15.978 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:15.978 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:15.978 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:15.978 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:15.978 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:00:15.978 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:15.979 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:15.967+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:15.979 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:15.992 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:16.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:00:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:16.079 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:16.079 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:16.080 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:16.071+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:16.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:16.180 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:16.174+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:16.234 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:16.234 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:16.240 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T13:00:16.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:16.280 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:16.274+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:16.372 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone23 from 0xc260508 (overlap 0x6beba5) to 0x2b2b0e 2024-09-16T13:00:16.381 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:16.371+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:16.392 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:16.393 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:16.407 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:16.481 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:16.474+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:16.506 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:16.506 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:16.515 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:16.571 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-16T13:00:16.582 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:16.578+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:16.611 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:16.611 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:16.622 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:16.683 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:16.675+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:16.725 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:16.725 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:16.735 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:16.784 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:16.778+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:16.877 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:16.877 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:16.885 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:16.875+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:16.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:16.973 INFO:teuthology.orchestra.run.smithi042.stdout:4392 trunc from 0xc260508 to 0x17a162b 2024-09-16T13:00:16.985 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:16.975+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:16.992 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:16.993 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:17.008 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:17.061 INFO:teuthology.orchestra.run.smithi042.stdout:4393 write 0x8968726 thru 0x89714aa (0x8d85 bytes) 2024-09-16T13:00:17.073 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:17.073 INFO:teuthology.orchestra.run.smithi042.stdout:4395 punch from 0x38198d6 to 0x381b3a4, (0x1ace bytes) 2024-09-16T13:00:17.073 INFO:teuthology.orchestra.run.smithi042.stdout:4396 flatten 2024-09-16T13:00:17.093 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:17.079+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:17.189 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:17.182+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:17.216 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:17.216 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:17.233 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:17.254 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T13:00:17.266 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:17.267 INFO:teuthology.orchestra.run.smithi042.stdout:4401 read 0x1dd27d2 thru 0x1ddc585 (0x9db4 bytes) 2024-09-16T13:00:17.270 INFO:teuthology.orchestra.run.smithi042.stdout:4402 read 0x447c8e3 thru 0x4480b5f (0x427d bytes) 2024-09-16T13:00:17.271 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:17.271 INFO:teuthology.orchestra.run.smithi042.stdout:4405 punch from 0x446d7f6 to 0x4471f11, (0x471b bytes) 2024-09-16T13:00:17.271 INFO:teuthology.orchestra.run.smithi042.stdout:4409 write 0x4f909f3 thru 0x4f91974 (0xf82 bytes) 2024-09-16T13:00:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:4410 write 0xf86316 thru 0xf90133 (0x9e1e bytes) 2024-09-16T13:00:17.289 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:17.282+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:17.295 INFO:teuthology.orchestra.run.smithi042.stdout:4411 write 0xa582fc7 thru 0xa58cbf3 (0x9c2d bytes) 2024-09-16T13:00:17.311 INFO:teuthology.orchestra.run.smithi042.stdout:4412 write 0x181eb93 thru 0x182e9bc (0xfe2a bytes) 2024-09-16T13:00:17.333 INFO:teuthology.orchestra.run.smithi042.stdout:4414 write 0xeb97b05 thru 0xeba218f (0xa68b bytes) 2024-09-16T13:00:17.334 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:17.334 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:17.344 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:17.350 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:17.350 INFO:teuthology.orchestra.run.smithi042.stdout:4417 trunc from 0xeba2190 to 0xe551f2d 2024-09-16T13:00:17.363 INFO:teuthology.orchestra.run.smithi042.stdout:4418 clone 25 order 25 su 262144 sc 10 2024-09-16T13:00:17.390 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:17.386+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:17.443 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:17.443 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:17.452 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:17.494 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:17.487+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:17.592 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:17.586+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:17.592 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:17.592 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:17.602 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:00:17.693 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:17.683+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:17.707 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:17.707 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:17.717 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:00:17.793 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:17.787+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:17.816 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:17.817 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:17.826 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:00:17.894 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:17.887+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:17.923 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:17.923 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:17.932 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:17.995 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:17.990+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:18.026 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:18.026 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:18.035 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:18.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:18.096 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:18.087+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:18.183 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:18.183 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:18.189 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:18.192 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:18.196 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:18.190+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:18.210 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:18.233 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:18.257 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:18.278 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:00:18.292 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:18.293 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:18.297 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:18.290+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:18.298 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:18.302 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:18.320 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:18.340 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:18.342 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:18.342 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:18.343 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:18.343 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:18.343 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:00:18.343 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:18.343 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:18.343 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:18.393 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:18.393 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:18.394 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:18.394 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:18.399 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:18.391+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:18.403 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:18.497 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:18.497 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:18.497 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:18.498 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:18.498 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:18.500 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:18.494+0000 7f2dcb806700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22331) UID: 0 2024-09-16T13:00:18.507 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:18.574 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-16T13:00:18.593 INFO:teuthology.orchestra.run.smithi042.stdout:4419 write 0xe447dd6 thru 0xe44f102 (0x732d bytes) 2024-09-16T13:00:18.601 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:18.594+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:18.614 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.246337293000003, 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T13:00:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5076 4.a 4:5041c5b5:::rbd_data.11b53fd98b75.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T13:00:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.451911+0000", 2024-09-16T13:00:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.158619622000003, 2024-09-16T13:00:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020443647999999998, 2024-09-16T13:00:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5076 2024-09-16T13:00:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451911+0000", 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451913+0000", 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451920+0000", 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5459999999999997e-06 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451938+0000", 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7581999999999999e-05 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.451950+0000", 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2033e-05 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452042+0000", 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2533000000000003e-05 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.452251+0000", 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020887 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.456761+0000", 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045099959999999996 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472274+0000", 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015512882 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472281+0000", 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3939999999999998e-06 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472324+0000", 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2389999999999999e-05 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.472355+0000", 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1436000000000002e-05 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5580 4.12 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.917496563999997, 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034176870999999998, 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5580 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693037+0000", 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0650000000000001e-06 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693045+0000", 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693070+0000", 2024-09-16T13:00:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5341999999999999e-05 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693081+0000", 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0611000000000001e-05 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693182+0000", 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100437 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693309+0000", 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012741400000000001 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.703090+0000", 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097814800000000004 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726908+0000", 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023817680000000001 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727023+0000", 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011449400000000001 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727130+0000", 2024-09-16T13:00:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010704099999999999 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727211+0000", 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1675000000000007e-05 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:18031 4.4 4:2649aabf:::rbd_data.11b51a3eb68b.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~5304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.450014060000001, 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94984657100000003, 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 18031 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160519+0000", 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3029999999999998e-06 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160536+0000", 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7181999999999999e-05 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160564+0000", 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971e-05 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160576+0000", 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160669+0000", 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3282999999999994e-05 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160680+0000", 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107188+0000", 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94650794999999999 2024-09-16T13:00:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107644+0000", 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00045604499999999997 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.108257+0000", 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061324700000000003 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.109811+0000", 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015538749999999999 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110175+0000", 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036419399999999998 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110285+0000", 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010938900000000001 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110364+0000", 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8689000000000005e-05 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55237 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.082567+0000", 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5279639820000002, 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017090014000000001, 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55237 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082567+0000", 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082567+0000", 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082568+0000", 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5599999999999995e-07 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082571+0000", 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.948e-06 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082576+0000", 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7180000000000001e-06 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.082580+0000", 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.597e-06 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.095399+0000", 2024-09-16T13:00:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012819179 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.095550+0000", 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150631 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.096161+0000", 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061149499999999999 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.097556+0000", 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013950779999999999 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.099583+0000", 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002026637 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.099626+0000", 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3231e-05 2024-09-16T13:00:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.099657+0000", 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0944000000000001e-05 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55315 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.185928+0000", 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4246027579999998, 2024-09-16T13:00:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017209391000000001, 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55315 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185928+0000", 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185928+0000", 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185929+0000", 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0109999999999999e-06 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185936+0000", 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8369999999999999e-06 2024-09-16T13:00:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185961+0000", 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4990999999999998e-05 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.185971+0000", 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5400000000000001e-06 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.197521+0000", 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011550685999999999 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.197655+0000", 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134041 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.198211+0000", 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000555238 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.200496+0000", 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022851640000000001 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.203024+0000", 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025280849999999998 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.203091+0000", 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7135e-05 2024-09-16T13:00:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.203138+0000", 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6662999999999997e-05 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55348 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.242380+0000", 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3681505850000004, 2024-09-16T13:00:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016534284, 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55348 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242380+0000", 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242380+0000", 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242381+0000", 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242383+0000", 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2879999999999999e-06 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242388+0000", 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.498e-06 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242391+0000", 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9239999999999999e-06 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.253143+0000", 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010752526 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.253264+0000", 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012047100000000001 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.253836+0000", 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057289099999999996 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.255835+0000", 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019988060000000001 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258786+0000", 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029510439999999999 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258864+0000", 2024-09-16T13:00:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7324999999999996e-05 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258915+0000", 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1093e-05 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55349 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.242396+0000", 2024-09-16T13:00:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3681346630000002, 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016602670999999999, 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55349 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242396+0000", 2024-09-16T13:00:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242396+0000", 2024-09-16T13:00:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242397+0000", 2024-09-16T13:00:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.58e-07 2024-09-16T13:00:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242399+0000", 2024-09-16T13:00:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3159999999999999e-06 2024-09-16T13:00:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242404+0000", 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7859999999999996e-06 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.242407+0000", 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8169999999999999e-06 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.254710+0000", 2024-09-16T13:00:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01230288 2024-09-16T13:00:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.254807+0000", 2024-09-16T13:00:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7310000000000002e-05 2024-09-16T13:00:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.255183+0000", 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037571600000000001 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.257088+0000", 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00190563 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258929+0000", 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018412140000000001 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258963+0000", 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3385e-05 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.258999+0000", 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6158999999999997e-05 2024-09-16T13:00:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55525 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.479709+0000", 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.1308221969999996, 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017363224999999999, 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55525 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479709+0000", 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479709+0000", 2024-09-16T13:00:18.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479709+0000", 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5600000000000002e-07 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479711+0000", 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7540000000000001e-06 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479714+0000", 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.298e-06 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.479718+0000", 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3529999999999997e-06 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.492361+0000", 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012642054 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.492421+0000", 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9993e-05 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.492684+0000", 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026362299999999999 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.494910+0000", 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022256789999999999 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.496953+0000", 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020428600000000001 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.497021+0000", 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8291000000000001e-05 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.497072+0000", 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1063999999999998e-05 2024-09-16T13:00:18.670 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55699 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8695724600000001, 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036277113999999999, 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.671 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55699 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740960+0000", 2024-09-16T13:00:18.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.48e-06 2024-09-16T13:00:18.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740967+0000", 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8499999999999996e-06 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740984+0000", 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7555999999999999e-05 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740991+0000", 2024-09-16T13:00:18.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6969999999999996e-06 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741042+0000", 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0927000000000002e-05 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741116+0000", 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4016000000000004e-05 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741422+0000", 2024-09-16T13:00:18.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030587600000000001 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.742556+0000", 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011342139999999999 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.776943+0000", 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034386727999999998 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.777160+0000", 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021688499999999999 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.777236+0000", 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5884999999999996e-05 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55733 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.677 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.822734+0000", 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7877973389999999, 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016635594, 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55733 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822734+0000", 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822734+0000", 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822734+0000", 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3500000000000002e-07 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822737+0000", 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1329999999999999e-06 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822743+0000", 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9259999999999997e-06 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822747+0000", 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.794e-06 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.834106+0000", 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011359088999999999 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.834191+0000", 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5131999999999997e-05 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.834584+0000", 2024-09-16T13:00:18.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039327299999999997 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.837181+0000", 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002596526 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839315+0000", 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021340320000000001 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839347+0000", 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2271000000000003e-05 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839369+0000", 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1982999999999999e-05 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55734 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.822751+0000", 2024-09-16T13:00:18.683 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7877802059999999, 2024-09-16T13:00:18.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016775394999999999, 2024-09-16T13:00:18.684 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.684 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.684 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55734 2024-09-16T13:00:18.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.685 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822751+0000", 2024-09-16T13:00:18.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822751+0000", 2024-09-16T13:00:18.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822751+0000", 2024-09-16T13:00:18.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.01e-07 2024-09-16T13:00:18.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822754+0000", 2024-09-16T13:00:18.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6230000000000001e-06 2024-09-16T13:00:18.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822760+0000", 2024-09-16T13:00:18.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:00:18.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.822763+0000", 2024-09-16T13:00:18.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.765e-06 2024-09-16T13:00:18.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.835261+0000", 2024-09-16T13:00:18.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012497291000000001 2024-09-16T13:00:18.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.835390+0000", 2024-09-16T13:00:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012971 2024-09-16T13:00:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.835924+0000", 2024-09-16T13:00:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053354900000000002 2024-09-16T13:00:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.837316+0000", 2024-09-16T13:00:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013921669999999999 2024-09-16T13:00:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839423+0000", 2024-09-16T13:00:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002106424 2024-09-16T13:00:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839482+0000", 2024-09-16T13:00:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9116000000000001e-05 2024-09-16T13:00:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.839526+0000", 2024-09-16T13:00:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4468000000000002e-05 2024-09-16T13:00:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55910 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.034703+0000", 2024-09-16T13:00:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5758277249999999, 2024-09-16T13:00:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015885804, 2024-09-16T13:00:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55910 2024-09-16T13:00:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.697 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034703+0000", 2024-09-16T13:00:18.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034703+0000", 2024-09-16T13:00:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034704+0000", 2024-09-16T13:00:18.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7999999999999995e-07 2024-09-16T13:00:18.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034711+0000", 2024-09-16T13:00:18.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4019999999999996e-06 2024-09-16T13:00:18.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034718+0000", 2024-09-16T13:00:18.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0940000000000002e-06 2024-09-16T13:00:18.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034723+0000", 2024-09-16T13:00:18.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2024-09-16T13:00:18.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.046578+0000", 2024-09-16T13:00:18.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011855286 2024-09-16T13:00:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.046683+0000", 2024-09-16T13:00:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104244 2024-09-16T13:00:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.047047+0000", 2024-09-16T13:00:18.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036465800000000002 2024-09-16T13:00:18.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.050476+0000", 2024-09-16T13:00:18.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034288449999999998 2024-09-16T13:00:18.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.050512+0000", 2024-09-16T13:00:18.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6032999999999999e-05 2024-09-16T13:00:18.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.050553+0000", 2024-09-16T13:00:18.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1226000000000001e-05 2024-09-16T13:00:18.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.050589+0000", 2024-09-16T13:00:18.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5790000000000001e-05 2024-09-16T13:00:18.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.706 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55911 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.034728+0000", 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5758033519999999, 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016693761000000001, 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55911 2024-09-16T13:00:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034728+0000", 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034728+0000", 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034728+0000", 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7800000000000001e-07 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034732+0000", 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1769999999999998e-06 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034738+0000", 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.034743+0000", 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4800000000000003e-06 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.048353+0000", 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013609804 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.048449+0000", 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6061000000000001e-05 2024-09-16T13:00:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.048844+0000", 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039503500000000002 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.050605+0000", 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017609360000000001 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.051361+0000", 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00075586999999999996 2024-09-16T13:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.051391+0000", 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0814e-05 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.051421+0000", 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0026999999999999e-05 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55952 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.531881507, 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016269499, 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55952 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078650+0000", 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0800000000000005e-07 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078653+0000", 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7499999999999999e-06 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078663+0000", 2024-09-16T13:00:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0455000000000001e-05 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.078667+0000", 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8770000000000003e-06 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.091010+0000", 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012342674999999999 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.091157+0000", 2024-09-16T13:00:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014694900000000001 2024-09-16T13:00:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.091652+0000", 2024-09-16T13:00:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049516700000000002 2024-09-16T13:00:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.094248+0000", 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025961970000000002 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.094801+0000", 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055320100000000002 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.094843+0000", 2024-09-16T13:00:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1353000000000002e-05 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.094919+0000", 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6366999999999996e-05 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:56410 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.565274+0000", 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.0452575039999998, 2024-09-16T13:00:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016007667, 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 56410 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565274+0000", 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565274+0000", 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565274+0000", 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2500000000000001e-07 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565277+0000", 2024-09-16T13:00:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5950000000000001e-06 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565283+0000", 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565287+0000", 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3519999999999998e-06 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.574749+0000", 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094626160000000001 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.574837+0000", 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8300000000000005e-05 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.575212+0000", 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000374166 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.579130+0000", 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039184670000000001 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.581142+0000", 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020120440000000002 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.581223+0000", 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1284000000000003e-05 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.581281+0000", 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7778999999999997e-05 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:56411 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.565290+0000", 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.0452414079999999, 2024-09-16T13:00:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016111232, 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 56411 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565290+0000", 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565290+0000", 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565290+0000", 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4999999999999998e-07 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565293+0000", 2024-09-16T13:00:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9110000000000002e-06 2024-09-16T13:00:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565298+0000", 2024-09-16T13:00:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6210000000000003e-06 2024-09-16T13:00:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565301+0000", 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.275e-06 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.576204+0000", 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010903079 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.576294+0000", 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0235999999999995e-05 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.576603+0000", 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030891499999999999 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.579856+0000", 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032527530000000002 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.581298+0000", 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001441711 2024-09-16T13:00:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.581352+0000", 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4323000000000001e-05 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.581401+0000", 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8958000000000001e-05 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:56412 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.565304+0000", 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.0452268419999999, 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018761489999999999, 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 56412 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565304+0000", 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565304+0000", 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565305+0000", 2024-09-16T13:00:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2300000000000002e-07 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565307+0000", 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.525e-06 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565315+0000", 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1689999999999992e-06 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.565319+0000", 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3730000000000001e-06 2024-09-16T13:00:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.577390+0000", 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012071215999999999 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.577495+0000", 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010522300000000001 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.577885+0000", 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038949399999999999 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.580535+0000", 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002650258 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.583856+0000", 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033211019999999998 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.583971+0000", 2024-09-16T13:00:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115487 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.584066+0000", 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4220000000000003e-05 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:56464 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.640849+0000", 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.969682132, 2024-09-16T13:00:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017332994000000001, 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 56464 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640849+0000", 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640849+0000", 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640850+0000", 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4000000000000001e-07 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640853+0000", 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6049999999999998e-06 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640862+0000", 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4970000000000001e-06 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640867+0000", 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1359999999999996e-06 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.651390+0000", 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010523406000000001 2024-09-16T13:00:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.651531+0000", 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140419 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.652450+0000", 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091971100000000003 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.656245+0000", 2024-09-16T13:00:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037945629999999999 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.657956+0000", 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001710753 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.658083+0000", 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000127126 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.658182+0000", 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9137999999999998e-05 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:56465 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:15.640873+0000", 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.9696582729999998, 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019943595000000001, 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 56465 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640873+0000", 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640873+0000", 2024-09-16T13:00:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640873+0000", 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5499999999999998e-07 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640877+0000", 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6940000000000001e-06 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640886+0000", 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6049999999999998e-06 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.640891+0000", 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2140000000000004e-06 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.653896+0000", 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013004742 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.654067+0000", 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017090200000000001 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.654848+0000", 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078153999999999995 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.656756+0000", 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019081510000000001 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.660686+0000", 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039300079999999996 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.660762+0000", 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6113999999999997e-05 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:15.660816+0000", 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.397e-05 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:18.749 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:18.749 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:18.749 INFO:teuthology.orchestra.run.smithi042.stdout:4421 trunc from 0xe551f2d to 0x9cd57bf 2024-09-16T13:00:18.750 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:18.751 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:18.746+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:18.801 INFO:teuthology.orchestra.run.smithi042.stdout:4422 read 0x761115a thru 0x7611380 (0x227 bytes) 2024-09-16T13:00:18.802 INFO:teuthology.orchestra.run.smithi042.stdout:4423 read 0x6756de2 thru 0x676224a (0xb469 bytes) 2024-09-16T13:00:18.802 INFO:teuthology.orchestra.run.smithi042.stdout:4424 write 0x1f85241 thru 0x1f859a4 (0x764 bytes) 2024-09-16T13:00:18.805 INFO:teuthology.orchestra.run.smithi042.stdout:4425 read 0x3f0d396 thru 0x3f157ef (0x845a bytes) 2024-09-16T13:00:18.825 INFO:teuthology.orchestra.run.smithi042.stdout:4426 write 0xe2f2d6b thru 0xe2f967b (0x6911 bytes) 2024-09-16T13:00:18.829 INFO:teuthology.orchestra.run.smithi042.stdout:4427 read 0xda4af9b thru 0xda4ea16 (0x3a7c bytes) 2024-09-16T13:00:18.830 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:18.830 INFO:teuthology.orchestra.run.smithi042.stdout:4429 read 0xd4256ca thru 0xd428517 (0x2e4e bytes) 2024-09-16T13:00:18.830 INFO:teuthology.orchestra.run.smithi042.stdout:4431 write 0x50bc9f4 thru 0x50c652a (0x9b37 bytes) 2024-09-16T13:00:18.834 INFO:teuthology.orchestra.run.smithi042.stdout:4432 punch from 0x8b110dd to 0x8b14e11, (0x3d34 bytes) 2024-09-16T13:00:18.834 INFO:teuthology.orchestra.run.smithi042.stdout:4434 write 0x9cabbfb thru 0x9cb3f8a (0x8390 bytes) 2024-09-16T13:00:18.838 INFO:teuthology.orchestra.run.smithi042.stdout:4436 writesame 0x15a2d83 thru 0x15b2c8d data_size 0x1(0xff0b bytes) 2024-09-16T13:00:18.851 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:18.843+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:18.883 INFO:teuthology.orchestra.run.smithi042.stdout:4437 trunc from 0xe2f967c to 0xca6b09c 2024-09-16T13:00:18.888 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:18.888 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:18.888 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:18.888 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:18.897 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:18.898 INFO:teuthology.orchestra.run.smithi042.stdout:4439 punch from 0x8bc36ae to 0x8bd2eb0, (0xf802 bytes) 2024-09-16T13:00:18.898 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:18.898 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:18.901 INFO:teuthology.orchestra.run.smithi042.stdout:4442 trunc from 0xca6b09c to 0xb962423 2024-09-16T13:00:18.915 INFO:teuthology.orchestra.run.smithi042.stdout:4444 write 0x4b57277 thru 0x4b5f235 (0x7fbf bytes) 2024-09-16T13:00:18.918 INFO:teuthology.orchestra.run.smithi042.stdout:4445 write 0x4b4332e thru 0x4b52b58 (0xf82b bytes) 2024-09-16T13:00:18.921 INFO:teuthology.orchestra.run.smithi042.stdout:4446 write 0x4c35dc thru 0x4c9c7e (0x66a3 bytes) 2024-09-16T13:00:18.924 INFO:teuthology.orchestra.run.smithi042.stdout:4447 writesame 0x345b3fc thru 0x345bf64 data_size 0x1(0xb69 bytes) 2024-09-16T13:00:18.928 INFO:teuthology.orchestra.run.smithi042.stdout:4449 writesame 0x10abcc5 thru 0x10b3fd7 data_size 0x5(0x8313 bytes) 2024-09-16T13:00:18.935 INFO:teuthology.orchestra.run.smithi042.stdout:4452 read 0x42adc64 thru 0x42b6124 (0x84c1 bytes) 2024-09-16T13:00:18.935 INFO:teuthology.orchestra.run.smithi042.stdout:4453 write 0x5bd6736 thru 0x5bdd88a (0x7155 bytes) 2024-09-16T13:00:18.939 INFO:teuthology.orchestra.run.smithi042.stdout:4454 read 0x88b5821 thru 0x88bd128 (0x7908 bytes) 2024-09-16T13:00:18.940 INFO:teuthology.orchestra.run.smithi042.stdout:4455 writesame 0xb3c5a0c thru 0xb3d3725 data_size 0x5b(0xdd1a bytes) 2024-09-16T13:00:18.944 INFO:teuthology.orchestra.run.smithi042.stdout:4456 punch from 0x2e34fa5 to 0x2e38dc3, (0x3e1e bytes) 2024-09-16T13:00:18.944 INFO:teuthology.orchestra.run.smithi042.stdout:4457 read 0xd70011 thru 0xd7eabd (0xeaad bytes) 2024-09-16T13:00:18.945 INFO:teuthology.orchestra.run.smithi042.stdout:4461 read 0x2f7101e thru 0x2f786cf (0x76b2 bytes) 2024-09-16T13:00:18.946 INFO:teuthology.orchestra.run.smithi042.stdout:4462 write 0x9204d05 thru 0x9211552 (0xc84e bytes) 2024-09-16T13:00:18.949 INFO:teuthology.orchestra.run.smithi042.stdout:4463 read 0x19c53e thru 0x1a7286 (0xad49 bytes) 2024-09-16T13:00:18.950 INFO:teuthology.orchestra.run.smithi042.stdout:4464 punch from 0x966cd2e to 0x9679f47, (0xd219 bytes) 2024-09-16T13:00:18.951 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:18.951 INFO:teuthology.orchestra.run.smithi042.stdout:4466 punch from 0x5e02611 to 0x5e102b9, (0xdca8 bytes) 2024-09-16T13:00:18.952 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:18.943+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:18.974 INFO:teuthology.orchestra.run.smithi042.stdout:4467 trunc from 0xb962423 to 0x4950b9 2024-09-16T13:00:18.991 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T13:00:18.991 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T13:00:18.991 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T13:00:18.991 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T13:00:18.991 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:00:18.994 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:18.994 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:18.994 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:18.994 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:18.995 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:19.004 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:19.053 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:19.046+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:19.060 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T13:00:19.060 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-16T13:00:19.060 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T13:00:19.060 DEBUG:teuthology.orchestra.run.smithi136:> type ceph-objectstore-tool 2024-09-16T13:00:19.065 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T13:00:19.065 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-16T13:00:19.145 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:19.145 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:19.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:19.145 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:19.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 93.590391500999999, 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:00:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:00:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.765354024999993, 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010976025, 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 45 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374760+0000", 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.13e-06 2024-09-16T13:00:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374769+0000", 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5699999999999999e-06 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374819+0000", 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9882000000000001e-05 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374828+0000", 2024-09-16T13:00:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382329+0000", 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075009009999999999 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382403+0000", 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4208999999999994e-05 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382514+0000", 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111276 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385533+0000", 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030190080000000001 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385564+0000", 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0926000000000002e-05 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385666+0000", 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101791 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385735+0000", 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8541999999999995e-05 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.765274454000007, 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:00:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:00:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:00:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:00:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:00:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:00:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:00:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.765227198000005, 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:00:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:00:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.765024261999997, 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:00:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.764993000000004, 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:00:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:00:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.764957237999994, 2024-09-16T13:00:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:00:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:00:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:00:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.764762607999998, 2024-09-16T13:00:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:00:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.764715762999998, 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:00:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.220439592000005, 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:00:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:00:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.220385684999997, 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:00:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.220343380000003, 2024-09-16T13:00:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:00:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.220184954999993, 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:00:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:00:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.220135474000003, 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:00:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:00:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:00:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.220004904000007, 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:00:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:00:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:00:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:00:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.219823391000006, 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:00:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:00:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 91.409295724000003, 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:00:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 91.409146770000007, 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64507 4.1e e83/31 4:7f7c7834:::rbd_data.11b5c2ee3542.0000000000000008:head v 83'225, mlcod=81'223)", 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.36973348299999997, 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021284401000000001, 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770380+0000", 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8999999999999998e-07 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777141+0000", 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067617670000000001 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777157+0000", 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777167+0000", 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777267+0000", 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100364 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777304+0000", 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7165000000000001e-05 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791601+0000", 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014297275999999999 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791664+0000", 2024-09-16T13:00:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2217000000000004e-05 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64509 4.15 e83/31 4:ad2e18f4:::rbd_data.11b5c2ee3542.0000000000000009:head v 83'183, mlcod=81'180)", 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.36297558099999999, 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014054395000000001, 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777138+0000", 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T13:00:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778534+0000", 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013956769999999999 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778555+0000", 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1046000000000001e-05 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778564+0000", 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9290000000000004e-06 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778633+0000", 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9363000000000007e-05 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778666+0000", 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135000000000003e-05 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791145+0000", 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012479743 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791192+0000", 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6156000000000003e-05 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:19.263 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:19.265 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:19.265 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T13:00:19.266 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:19.259+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:19.359 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:19.360 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:19.366 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:19.362+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:19.369 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:19.461 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:19.462 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:19.467 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:19.462+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:19.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:19.568 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:19.559+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:19.571 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:19.571 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:19.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:19.571 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:19.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.571 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.571 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:19.571 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8719182520000004, 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8719032259999997, 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8718872040000001, 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8718711079999997, 2024-09-16T13:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8718552180000003, 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8718395479999996, 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8718238009999997, 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8718080129999999, 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8717926909999996, 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8717727660000003, 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8717568179999997, 2024-09-16T13:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8717416570000003, 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8717260089999996, 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8717109120000002, 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8716956500000004, 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8716799039999996, 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8716643069999996, 2024-09-16T13:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8716490029999999, 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8716335649999998, 2024-09-16T13:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:00:19.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.871617852, 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:19.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:19.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:00:19.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:00:19.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.671 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:19.671 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:19.671 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:19.673 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:19.673 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:19.663+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:19.773 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:19.766+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:19.820 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:19.820 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:19.820 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:19.820 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:19.829 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:19.874 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:19.870+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:19.931 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:19.931 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:19.931 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:19.931 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:19.931 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:19.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:19.986 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:19.979+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:20.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T13:00:20.034+0000 7f9550397300 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T13:00:20.044 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54893 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.678970+0000", 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3611414809999998, 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056907577000000001, 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54893 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678970+0000", 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678970+0000", 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678970+0000", 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.27e-07 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678976+0000", 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8710000000000002e-06 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678984+0000", 2024-09-16T13:00:20.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5220000000000007e-06 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678988+0000", 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9149999999999998e-06 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.735622+0000", 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056633731 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.735717+0000", 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4658000000000001e-05 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.735877+0000", 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016045299999999999 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54894 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.049 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.678993+0000", 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3611183980000003, 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057108937999999998, 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54894 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678993+0000", 2024-09-16T13:00:20.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678993+0000", 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678993+0000", 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678995+0000", 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679004+0000", 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9250000000000001e-06 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679008+0000", 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8510000000000001e-06 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.735926+0000", 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056917945999999997 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736046+0000", 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120673 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736102+0000", 2024-09-16T13:00:20.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.516e-05 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54895 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 831488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679012+0000", 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.361098922, 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057255952999999998, 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54895 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679012+0000", 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679012+0000", 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679013+0000", 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2800000000000002e-07 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679015+0000", 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679023+0000", 2024-09-16T13:00:20.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.799e-06 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679027+0000", 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6789999999999998e-06 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736130+0000", 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057102922 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736192+0000", 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2439000000000003e-05 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736268+0000", 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5717999999999996e-05 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54896 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679031+0000", 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3610798590000002, 2024-09-16T13:00:20.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057360204999999997, 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54896 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679031+0000", 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679031+0000", 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679032+0000", 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.39e-07 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679034+0000", 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0389999999999999e-06 2024-09-16T13:00:20.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679042+0000", 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4440000000000008e-06 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679046+0000", 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.721e-06 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736301+0000", 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057254699999999999 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736339+0000", 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8424999999999999e-05 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736391+0000", 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2437e-05 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.063 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54897 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 897024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.063 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679050+0000", 2024-09-16T13:00:20.063 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3610611419999996, 2024-09-16T13:00:20.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057471563000000003, 2024-09-16T13:00:20.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54897 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679050+0000", 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679050+0000", 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679050+0000", 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679052+0000", 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0980000000000001e-06 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679061+0000", 2024-09-16T13:00:20.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0330000000000008e-06 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679064+0000", 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8160000000000004e-06 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736415+0000", 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05735109 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736462+0000", 2024-09-16T13:00:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.702e-05 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736522+0000", 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9094e-05 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54898 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.067 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679069+0000", 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3610425880000001, 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057588469000000003, 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54898 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679069+0000", 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679069+0000", 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679069+0000", 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679071+0000", 2024-09-16T13:00:20.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0559999999999999e-06 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679077+0000", 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.964e-06 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679081+0000", 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7500000000000001e-06 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736551+0000", 2024-09-16T13:00:20.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057470585999999997 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736596+0000", 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4725000000000003e-05 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736657+0000", 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0956999999999999e-05 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54899 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679084+0000", 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3610266119999999, 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057724701000000003, 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54899 2024-09-16T13:00:20.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679084+0000", 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679084+0000", 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679085+0000", 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6199999999999999e-07 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679087+0000", 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8419999999999999e-06 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679095+0000", 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0919999999999998e-06 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679102+0000", 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3540000000000002e-06 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736687+0000", 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057585166 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736737+0000", 2024-09-16T13:00:20.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9872e-05 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736809+0000", 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2013000000000003e-05 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54900 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679107+0000", 2024-09-16T13:00:20.076 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3610045990000001, 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057834479000000001, 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54900 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679107+0000", 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679107+0000", 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679107+0000", 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6600000000000002e-07 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679109+0000", 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1449999999999998e-06 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679117+0000", 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1440000000000003e-06 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679121+0000", 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.653e-06 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736842+0000", 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057721040000000001 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736888+0000", 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6581e-05 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736941+0000", 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2549999999999997e-05 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54901 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679125+0000", 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3609861749999999, 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057944962000000003, 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54901 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679125+0000", 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679125+0000", 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679125+0000", 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:00:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679127+0000", 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.013e-06 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679136+0000", 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679139+0000", 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.659e-06 2024-09-16T13:00:20.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736973+0000", 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057833415999999999 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737017+0000", 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4427e-05 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737070+0000", 2024-09-16T13:00:20.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2855000000000001e-05 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54902 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679143+0000", 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3609677739999997, 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058058212999999997, 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54902 2024-09-16T13:00:20.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.086 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679143+0000", 2024-09-16T13:00:20.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679143+0000", 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679144+0000", 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679146+0000", 2024-09-16T13:00:20.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9649999999999998e-06 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679154+0000", 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2239999999999995e-06 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679157+0000", 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4929999999999998e-06 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737101+0000", 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057943698000000002 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737149+0000", 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7453999999999997e-05 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737202+0000", 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2994000000000001e-05 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54903 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679161+0000", 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3609497470000003, 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145532, 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54903 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679161+0000", 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679161+0000", 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679162+0000", 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5499999999999998e-07 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679164+0000", 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0849999999999999e-06 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679170+0000", 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679173+0000", 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6729999999999998e-06 2024-09-16T13:00:20.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737217+0000", 2024-09-16T13:00:20.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058043933999999998 2024-09-16T13:00:20.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737253+0000", 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6056000000000002e-05 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737307+0000", 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3440000000000003e-05 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54904 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1126400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.097 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679177+0000", 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3609336660000002, 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058235595000000001, 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54904 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679177+0000", 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679177+0000", 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679178+0000", 2024-09-16T13:00:20.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679180+0000", 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679188+0000", 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.585e-06 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679192+0000", 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8920000000000002e-06 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737321+0000", 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058129382 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737356+0000", 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4851999999999998e-05 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737413+0000", 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6716999999999999e-05 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54905 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679196+0000", 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.360915264, 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058328532000000002, 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54905 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679196+0000", 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679196+0000", 2024-09-16T13:00:20.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679196+0000", 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679217+0000", 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0645e-05 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679228+0000", 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0728e-05 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679232+0000", 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7990000000000001e-06 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737428+0000", 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058195445999999998 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737471+0000", 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3585999999999999e-05 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737524+0000", 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2932000000000001e-05 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54906 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1191936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679237+0000", 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3608736700000001, 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442112999999997, 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54906 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679237+0000", 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679237+0000", 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679238+0000", 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-07 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679240+0000", 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2369999999999999e-06 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679249+0000", 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999999e-06 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679252+0000", 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6710000000000001e-06 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737539+0000", 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058286642 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737586+0000", 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6900999999999997e-05 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737680+0000", 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3604999999999996e-05 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54907 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1224704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679257+0000", 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.360853852, 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058610582000000001, 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54907 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679257+0000", 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679257+0000", 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679258+0000", 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679260+0000", 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9640000000000002e-06 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679268+0000", 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6039999999999993e-06 2024-09-16T13:00:20.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679272+0000", 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8829999999999999e-06 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737712+0000", 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439921999999998 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737797+0000", 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4739000000000003e-05 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737868+0000", 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1069999999999995e-05 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54908 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.115 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679276+0000", 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3608347030000001, 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058697892000000002, 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54908 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679276+0000", 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679276+0000", 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679277+0000", 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7300000000000002e-07 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679279+0000", 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679285+0000", 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7679999999999997e-06 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679288+0000", 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.568e-06 2024-09-16T13:00:20.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737884+0000", 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058596033999999998 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737923+0000", 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8659999999999999e-05 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737974+0000", 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1520999999999997e-05 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54909 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679292+0000", 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3608193990000004, 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058833353999999997, 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54909 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679292+0000", 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679292+0000", 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679292+0000", 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679294+0000", 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.888e-06 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679302+0000", 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3559999999999993e-06 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679306+0000", 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3500000000000001e-06 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737989+0000", 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058683524000000001 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738029+0000", 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9854999999999997e-05 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738125+0000", 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6003999999999998e-05 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54910 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1323008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679310+0000", 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3608013979999996, 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059166712000000003, 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54910 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679310+0000", 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679310+0000", 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679310+0000", 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.34e-07 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679312+0000", 2024-09-16T13:00:20.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0219999999999999e-06 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679321+0000", 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3629999999999994e-06 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679324+0000", 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.738e-06 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738187+0000", 2024-09-16T13:00:20.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058862652000000001 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738318+0000", 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013088299999999999 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738476+0000", 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015862 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54911 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1355776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679328+0000", 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.360782639, 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059496683000000002, 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54911 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679328+0000", 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679328+0000", 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679329+0000", 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679331+0000", 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9069999999999999e-06 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679339+0000", 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7469999999999994e-06 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679343+0000", 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.749e-06 2024-09-16T13:00:20.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738522+0000", 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059178423000000001 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738658+0000", 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000136766 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738825+0000", 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016672099999999999 2024-09-16T13:00:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54912 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1388544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679358+0000", 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.3607531970000002, 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060353395999999997, 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54912 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679358+0000", 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679358+0000", 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679358+0000", 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679360+0000", 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.108e-06 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679369+0000", 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6850000000000007e-06 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679373+0000", 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.895e-06 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.739469+0000", 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060096019000000001 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.739561+0000", 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1526000000000004e-05 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.739711+0000", 2024-09-16T13:00:20.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015069899999999999 2024-09-16T13:00:20.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.137 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:20.138 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:20.139 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:20.131+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:20.239 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:20.231+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:20.248 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:20.248 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:20.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:20.281 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T13:00:20.288 INFO:teuthology.orchestra.run.smithi042.stdout:4468 write 0xbb6935a thru 0xbb6b7af (0x2456 bytes) 2024-09-16T13:00:20.290 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.301 INFO:teuthology.orchestra.run.smithi042.stdout:4470 writesame 0xdb6b7f9 thru 0xdb7008c data_size 0xa(0x4894 bytes) 2024-09-16T13:00:20.303 INFO:teuthology.orchestra.run.smithi042.stdout:4471 write 0x6f7fd43 thru 0x6f85010 (0x52ce bytes) 2024-09-16T13:00:20.317 INFO:teuthology.orchestra.run.smithi042.stdout:4472 write 0xeb531ea thru 0xeb59757 (0x656e bytes) 2024-09-16T13:00:20.320 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.329 INFO:teuthology.orchestra.run.smithi042.stdout:4474 trunc from 0xeb59758 to 0xa34cee4 2024-09-16T13:00:20.340 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:20.331+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:20.343 INFO:teuthology.orchestra.run.smithi042.stdout:4476 punch from 0x34a24cf to 0x34ade59, (0xb98a bytes) 2024-09-16T13:00:20.343 INFO:teuthology.orchestra.run.smithi042.stdout:4478 punch from 0x8ee4730 to 0x8eed51d, (0x8ded bytes) 2024-09-16T13:00:20.343 INFO:teuthology.orchestra.run.smithi042.stdout:4479 write 0x5566225 thru 0x556d3a7 (0x7183 bytes) 2024-09-16T13:00:20.345 INFO:teuthology.orchestra.run.smithi042.stdout:4480 read 0x865f033 thru 0x8664d14 (0x5ce2 bytes) 2024-09-16T13:00:20.348 INFO:teuthology.orchestra.run.smithi042.stdout:4482 trunc from 0xa34cee4 to 0xc1db7c6 2024-09-16T13:00:20.354 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:20.356 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.356 INFO:teuthology.orchestra.run.smithi042.stdout:4486 writesame 0x69cfcc2 thru 0x69d3831 data_size 0x18(0x3b70 bytes) 2024-09-16T13:00:20.372 INFO:teuthology.orchestra.run.smithi042.stdout:4488 trunc from 0xc1db7c6 to 0x4c14f8e 2024-09-16T13:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout:4489 write 0xc725b35 thru 0xc72fe43 (0xa30f bytes) 2024-09-16T13:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout:4490 read 0x9ee2b95 thru 0x9ef137d (0xe7e9 bytes) 2024-09-16T13:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout:4492 write 0x58c5b17 thru 0x58d105f (0xb549 bytes) 2024-09-16T13:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout:4493 write 0x34075fd thru 0x34134b7 (0xbebb bytes) 2024-09-16T13:00:20.408 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout:4495 punch from 0x8feec45 to 0x8ff433d, (0x56f8 bytes) 2024-09-16T13:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout:4496 writesame 0x572eecd thru 0x5732c3e data_size 0x41(0x3d72 bytes) 2024-09-16T13:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout:4497 read 0x33d5cc7 thru 0x33e33ff (0xd739 bytes) 2024-09-16T13:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout:4498 writesame 0xaf613ed thru 0xaf6e862 data_size 0x15(0xd476 bytes) 2024-09-16T13:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout:4499 writesame 0x1b16b23 thru 0x1b18840 data_size 0x1(0x1d1e bytes) 2024-09-16T13:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout:4500 read 0x1e7bb5b thru 0x1e81452 (0x58f8 bytes) 2024-09-16T13:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout:4501 writesame 0x144edd6 thru 0x145413a data_size 0x1(0x5365 bytes) 2024-09-16T13:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout:4502 write 0x8edab4c thru 0x8ee9b13 (0xefc8 bytes) 2024-09-16T13:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout:4503 punch from 0x24d6448 to 0x24e1aff, (0xb6b7 bytes) 2024-09-16T13:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout:4504 write 0x1767440 thru 0x176d7d6 (0x6397 bytes) 2024-09-16T13:00:20.441 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:20.434+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout:4505 read 0x35b6990 thru 0x35bb062 (0x46d3 bytes) 2024-09-16T13:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout:4506 writesame 0xcb1d5b thru 0xcb5c44 data_size 0x1(0x3eea bytes) 2024-09-16T13:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout:4507 writesame 0x34810c7 thru 0x3487087 data_size 0x1(0x5fc1 bytes) 2024-09-16T13:00:20.484 INFO:teuthology.orchestra.run.smithi042.stdout:4508 write 0xe604a75 thru 0xe60c3a0 (0x792c bytes) 2024-09-16T13:00:20.485 INFO:teuthology.orchestra.run.smithi042.stdout:4509 write 0x33bec28 thru 0x33c02c6 (0x169f bytes) 2024-09-16T13:00:20.488 INFO:teuthology.orchestra.run.smithi042.stdout:4510 write 0x9fab17 thru 0x9fb701 (0xbeb bytes) 2024-09-16T13:00:20.490 INFO:teuthology.orchestra.run.smithi042.stdout:4511 write 0xa0a0f65 thru 0xa0b09e3 (0xfa7f bytes) 2024-09-16T13:00:20.492 INFO:teuthology.orchestra.run.smithi042.stdout:4513 write 0x2a23c71 thru 0x2a2bd48 (0x80d8 bytes) 2024-09-16T13:00:20.494 INFO:teuthology.orchestra.run.smithi042.stdout:4514 punch from 0x4a18d78 to 0x4a26c43, (0xdecb bytes) 2024-09-16T13:00:20.494 INFO:teuthology.orchestra.run.smithi042.stdout:4515 read 0x9a9194c thru 0x9a94283 (0x2938 bytes) 2024-09-16T13:00:20.495 INFO:teuthology.orchestra.run.smithi042.stdout:4516 punch from 0x23765f7 to 0x23834f5, (0xcefe bytes) 2024-09-16T13:00:20.495 INFO:teuthology.orchestra.run.smithi042.stdout:4517 read 0x421d1eb thru 0x421ebf1 (0x1a07 bytes) 2024-09-16T13:00:20.495 INFO:teuthology.orchestra.run.smithi042.stdout:4518 read 0xd4e8718 thru 0xd4e9286 (0xb6f bytes) 2024-09-16T13:00:20.497 INFO:teuthology.orchestra.run.smithi042.stdout:4521 trunc from 0xe60c3a1 to 0xed5f1c0 2024-09-16T13:00:20.504 INFO:teuthology.orchestra.run.smithi042.stdout:4522 read 0xb204ade thru 0xb20fdd5 (0xb2f8 bytes) 2024-09-16T13:00:20.505 INFO:teuthology.orchestra.run.smithi042.stdout:4523 read 0xd089f14 thru 0xd0963c1 (0xc4ae bytes) 2024-09-16T13:00:20.505 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:20.505 INFO:teuthology.orchestra.run.smithi042.stdout:4524 write 0x568ed81 thru 0x569177d (0x29fd bytes) 2024-09-16T13:00:20.508 INFO:teuthology.orchestra.run.smithi042.stdout:4526 write 0x523ff47 thru 0x524edc4 (0xee7e bytes) 2024-09-16T13:00:20.512 INFO:teuthology.orchestra.run.smithi042.stdout:4527 punch from 0xc81af5e to 0xc8273d2, (0xc474 bytes) 2024-09-16T13:00:20.512 INFO:teuthology.orchestra.run.smithi042.stdout:4528 punch from 0x159e9a4 to 0x15a6334, (0x7990 bytes) 2024-09-16T13:00:20.513 INFO:teuthology.orchestra.run.smithi042.stdout:4529 trunc from 0xed5f1c0 to 0xdd6462b 2024-09-16T13:00:20.517 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54223 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8560264479999997, 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028808500000000001, 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54223 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655799+0000", 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655801+0000", 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6579999999999998e-06 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655811+0000", 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5070000000000006e-06 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655816+0000", 2024-09-16T13:00:20.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8960000000000002e-06 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684450+0000", 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028634057000000001 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684521+0000", 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1453999999999997e-05 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684607+0000", 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5447999999999999e-05 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54224 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1220608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8560036459999996, 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028979718000000002, 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54224 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655822+0000", 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2600000000000002e-07 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655824+0000", 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.593e-06 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655833+0000", 2024-09-16T13:00:20.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2099999999999999e-06 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655838+0000", 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5650000000000003e-06 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684637+0000", 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028798550999999999 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684715+0000", 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8242000000000001e-05 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684801+0000", 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6031000000000004e-05 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54225 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8559815290000001, 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029147567999999999, 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54225 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:20.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655844+0000", 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.06e-07 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655846+0000", 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5059999999999998e-06 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655856+0000", 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.825e-06 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655861+0000", 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6990000000000002e-06 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684830+0000", 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028969385 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684907+0000", 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6800999999999999e-05 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684991+0000", 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3845999999999996e-05 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54226 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8559585890000001, 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029452944000000002, 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54226 2024-09-16T13:00:20.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655867+0000", 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8599999999999998e-07 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655869+0000", 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5849999999999998e-06 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655878+0000", 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2620000000000004e-06 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655883+0000", 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7350000000000004e-06 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685017+0000", 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029134083000000002 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685096+0000", 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999994e-05 2024-09-16T13:00:20.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685319+0000", 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022352299999999999 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54227 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1318912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8559361020000003, 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029655159, 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54227 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:20.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5700000000000002e-07 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655892+0000", 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.486e-06 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655903+0000", 2024-09-16T13:00:20.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1e-05 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655907+0000", 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.831e-06 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685347+0000", 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029439844 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685416+0000", 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8561999999999998e-05 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685544+0000", 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012787900000000001 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54228 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8559117570000003, 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029867089999999999, 2024-09-16T13:00:20.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54228 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2499999999999995e-07 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655916+0000", 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9170000000000002e-06 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655926+0000", 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7769999999999997e-06 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655931+0000", 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8470000000000003e-06 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685571+0000", 2024-09-16T13:00:20.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029640061999999998 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685642+0000", 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1327000000000003e-05 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685780+0000", 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013763499999999999 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54229 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8558880499999999, 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030082340999999999, 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54229 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1500000000000005e-07 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655940+0000", 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5840000000000002e-06 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655949+0000", 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9979999999999992e-06 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655954+0000", 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.746e-06 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685808+0000", 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029854776999999999 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685879+0000", 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0593000000000007e-05 2024-09-16T13:00:20.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.686019+0000", 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000140128 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54832 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 192512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:20.548 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8092348850000004, 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.032379022, 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54832 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4099999999999999e-07 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702592+0000", 2024-09-16T13:00:20.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.995e-06 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702597+0000", 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2120000000000002e-06 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702600+0000", 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9919999999999999e-06 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.727494+0000", 2024-09-16T13:00:20.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024893141000000001 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.727564+0000", 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0761999999999998e-05 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734969+0000", 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0074044790000000003 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54858 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1044480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8079165049999997, 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028959381999999999, 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54858 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703909+0000", 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7599999999999997e-07 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703911+0000", 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4789999999999999e-06 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703917+0000", 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2389999999999999e-06 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703920+0000", 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.021e-06 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732697+0000", 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028776665 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732778+0000", 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1018000000000003e-05 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732868+0000", 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9483999999999999e-05 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54859 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1077248~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8078990700000004, 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029138931999999999, 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54859 2024-09-16T13:00:20.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9400000000000001e-07 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703928+0000", 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1179999999999999e-06 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703934+0000", 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.84e-06 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703938+0000", 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5219999999999999e-06 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732893+0000", 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028955005999999998 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732974+0000", 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1569000000000005e-05 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733065+0000", 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0482999999999994e-05 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54860 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1110016~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8078829909999996, 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029318429999999999, 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54860 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703944+0000", 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9209999999999999e-06 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704020+0000", 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6217999999999994e-05 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704027+0000", 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3559999999999998e-06 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733091+0000", 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029064349999999999 2024-09-16T13:00:20.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733167+0000", 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6058999999999998e-05 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733260+0000", 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3110999999999997e-05 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54861 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8077918000000004, 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029431589000000001, 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54861 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704034+0000", 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6199999999999997e-07 2024-09-16T13:00:20.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704037+0000", 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9100000000000001e-06 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704045+0000", 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2150000000000001e-06 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704048+0000", 2024-09-16T13:00:20.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6380000000000002e-06 2024-09-16T13:00:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733285+0000", 2024-09-16T13:00:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029236999999999999 2024-09-16T13:00:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733365+0000", 2024-09-16T13:00:20.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9461000000000004e-05 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733465+0000", 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9603000000000002e-05 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54862 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1175552~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8077721279999999, 2024-09-16T13:00:20.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029595202000000001, 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54862 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3300000000000002e-07 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704055+0000", 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0940000000000002e-06 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704061+0000", 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.835e-06 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704064+0000", 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.948e-06 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733485+0000", 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029420565999999999 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733552+0000", 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7409999999999993e-05 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733648+0000", 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5816000000000006e-05 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54863 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1208320~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:20.574 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8077573300000003, 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029759681, 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54863 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704068+0000", 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704076+0000", 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0299999999999994e-06 2024-09-16T13:00:20.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704082+0000", 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.755e-06 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704085+0000", 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1159999999999999e-06 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733670+0000", 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029585163000000001 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733742+0000", 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2069000000000004e-05 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733827+0000", 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5161000000000007e-05 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54864 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8077362059999995, 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927993, 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54864 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.579 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:20.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3599999999999999e-07 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704091+0000", 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-06 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704096+0000", 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0950000000000003e-06 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704099+0000", 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8329999999999998e-06 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733853+0000", 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029754368999999999 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733931+0000", 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7956999999999999e-05 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734017+0000", 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5272999999999995e-05 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54865 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1273856~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8077222729999995, 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030108264999999999, 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54865 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704103+0000", 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704104+0000", 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.716e-06 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704110+0000", 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4410000000000001e-06 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704113+0000", 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1839999999999999e-06 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734040+0000", 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927013999999998 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734122+0000", 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1481000000000001e-05 2024-09-16T13:00:20.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734211+0000", 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9140000000000004e-05 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54866 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1306624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.807707862, 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030274079999999998, 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.587 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54866 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704119+0000", 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9870000000000002e-06 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704125+0000", 2024-09-16T13:00:20.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3410000000000002e-06 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704133+0000", 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734225+0000", 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030092026000000001 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734306+0000", 2024-09-16T13:00:20.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0634000000000001e-05 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734391+0000", 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5082999999999998e-05 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54867 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.807687789, 2024-09-16T13:00:20.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030392758999999998, 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54867 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704139+0000", 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.99e-06 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704145+0000", 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704148+0000", 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.078e-06 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734412+0000", 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030264287000000001 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734469+0000", 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6925999999999997e-05 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734530+0000", 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0381999999999999e-05 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54868 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1372160~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8076730940000001, 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030530140000000001, 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54868 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704245+0000", 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3037999999999999e-05 2024-09-16T13:00:20.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704256+0000", 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0718e-05 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704261+0000", 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734544+0000", 2024-09-16T13:00:20.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030283929000000001 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734595+0000", 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0356999999999999e-05 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734682+0000", 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6938e-05 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54869 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1404928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.8075588539999998, 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030584762000000001, 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54869 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:20.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704267+0000", 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4700000000000001e-07 2024-09-16T13:00:20.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704269+0000", 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1390000000000002e-06 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704275+0000", 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9449999999999998e-06 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704278+0000", 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1429999999999998e-06 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734705+0000", 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030427551000000001 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734778+0000", 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2502000000000005e-05 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734851+0000", 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2835000000000003e-05 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:20.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:20.604 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:20.604 INFO:teuthology.orchestra.run.smithi042.stdout:4531 write 0x3dc1905 thru 0x3dc3925 (0x2021 bytes) 2024-09-16T13:00:20.605 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:20.606 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:20.606 INFO:teuthology.orchestra.run.smithi042.stdout:4533 trunc from 0xdd6462b to 0x91e09cd 2024-09-16T13:00:20.606 INFO:teuthology.orchestra.run.smithi042.stdout:4535 write 0x5aba23d thru 0x5abba64 (0x1828 bytes) 2024-09-16T13:00:20.606 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:20.607 INFO:teuthology.orchestra.run.smithi042.stdout:4536 writesame 0xba489e1 thru 0xba4f4ce data_size 0x1(0x6aee bytes) 2024-09-16T13:00:20.607 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.607 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.607 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.607 INFO:teuthology.orchestra.run.smithi042.stdout:4541 write 0x3427e22 thru 0x3434c81 (0xce60 bytes) 2024-09-16T13:00:20.607 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:20.608 INFO:teuthology.orchestra.run.smithi042.stdout:4542 read 0x84c0c7 thru 0x855612 (0x954c bytes) 2024-09-16T13:00:20.608 INFO:teuthology.orchestra.run.smithi042.stdout:4543 trunc from 0xba4f4cf to 0x496cf8b 2024-09-16T13:00:20.608 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:20.608 INFO:teuthology.orchestra.run.smithi042.stdout:4545 punch from 0x10effb0 to 0x10fb3a9, (0xb3f9 bytes) 2024-09-16T13:00:20.608 INFO:teuthology.orchestra.run.smithi042.stdout:4546 writesame 0xb7a437 thru 0xb80968 data_size 0x1(0x6532 bytes) 2024-09-16T13:00:20.609 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:20.598+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:20.617 INFO:teuthology.orchestra.run.smithi042.stdout:4548 write 0x39f18aa thru 0x39ff57a (0xdcd1 bytes) 2024-09-16T13:00:20.619 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:20.638 INFO:teuthology.orchestra.run.smithi042.stdout:4549 write 0x94a49b0 thru 0x94a896e (0x3fbf bytes) 2024-09-16T13:00:20.641 INFO:teuthology.orchestra.run.smithi042.stdout:4551 read 0x86ebdf2 thru 0x86f7204 (0xb413 bytes) 2024-09-16T13:00:20.642 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.642 INFO:teuthology.orchestra.run.smithi042.stdout:4555 punch from 0x3f4b3a7 to 0x3f4ba55, (0x6ae bytes) 2024-09-16T13:00:20.642 INFO:teuthology.orchestra.run.smithi042.stdout:4556 read 0x937193e thru 0x937a784 (0x8e47 bytes) 2024-09-16T13:00:20.647 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:20.648 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:20.648 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:20.648 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:20.648 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:20.648 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:20.648 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:20.648 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:20.648 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:20.651 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:20.651 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:20.656 INFO:teuthology.orchestra.run.smithi042.stdout:4557 trunc from 0x94a896f to 0x25321a6 2024-09-16T13:00:20.683 INFO:teuthology.orchestra.run.smithi042.stdout:4558 write 0x70b456c thru 0x70be2b0 (0x9d45 bytes) 2024-09-16T13:00:20.686 INFO:teuthology.orchestra.run.smithi042.stdout:4560 read 0x40ad130 thru 0x40b99ef (0xc8c0 bytes) 2024-09-16T13:00:20.687 INFO:teuthology.orchestra.run.smithi042.stdout:4561 read 0x508c7ea thru 0x5096f61 (0xa778 bytes) 2024-09-16T13:00:20.687 INFO:teuthology.orchestra.run.smithi042.stdout:4562 write 0x706d8df thru 0x707b5c2 (0xdce4 bytes) 2024-09-16T13:00:20.691 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.694 INFO:teuthology.orchestra.run.smithi042.stdout:4564 trunc from 0x70be2b1 to 0x8c08621 2024-09-16T13:00:20.703 INFO:teuthology.orchestra.run.smithi042.stdout:4565 write 0x8517460 thru 0x8526a06 (0xf5a7 bytes) 2024-09-16T13:00:20.705 INFO:teuthology.orchestra.run.smithi042.stdout:4566 read 0x2b9fd69 thru 0x2ba16c1 (0x1959 bytes) 2024-09-16T13:00:20.706 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:20.698+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:20.715 INFO:teuthology.orchestra.run.smithi042.stdout:4568 trunc from 0x8c08621 to 0x3996a9a 2024-09-16T13:00:20.727 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:20.727 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:20.739 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:20.751 INFO:teuthology.orchestra.run.smithi042.stdout:4569 write 0xba89311 thru 0xba8efff (0x5cef bytes) 2024-09-16T13:00:20.753 INFO:teuthology.orchestra.run.smithi042.stdout:4570 write 0x79b52f thru 0x7a0ffa (0x5acc bytes) 2024-09-16T13:00:20.756 INFO:teuthology.orchestra.run.smithi042.stdout:4572 punch from 0xb44c818 to 0xb459a6d, (0xd255 bytes) 2024-09-16T13:00:20.756 INFO:teuthology.orchestra.run.smithi042.stdout:4573 writesame 0x785d022 thru 0x785e628 data_size 0x1(0x1607 bytes) 2024-09-16T13:00:20.761 INFO:teuthology.orchestra.run.smithi042.stdout:4574 write 0x7175924 thru 0x7180a2f (0xb10c bytes) 2024-09-16T13:00:20.764 INFO:teuthology.orchestra.run.smithi042.stdout:4576 trunc from 0xba8f000 to 0xba99bbe 2024-09-16T13:00:20.772 INFO:teuthology.orchestra.run.smithi042.stdout:4577 write 0x6e394e7 thru 0x6e42c02 (0x971c bytes) 2024-09-16T13:00:20.775 INFO:teuthology.orchestra.run.smithi042.stdout:4578 write 0xb2d52e0 thru 0xb2dd4fa (0x821b bytes) 2024-09-16T13:00:20.778 INFO:teuthology.orchestra.run.smithi042.stdout:4579 write 0xb8350fd thru 0xb83ce35 (0x7d39 bytes) 2024-09-16T13:00:20.781 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.781 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.781 INFO:teuthology.orchestra.run.smithi042.stdout:4582 read 0x173ec8a thru 0x1745dc3 (0x713a bytes) 2024-09-16T13:00:20.781 INFO:teuthology.orchestra.run.smithi042.stdout:4583 write 0x6cd611e thru 0x6cd9ade (0x39c1 bytes) 2024-09-16T13:00:20.784 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.784 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.784 INFO:teuthology.orchestra.run.smithi042.stdout:4587 read 0x925f991 thru 0x926a5e4 (0xac54 bytes) 2024-09-16T13:00:20.784 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.785 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.785 INFO:teuthology.orchestra.run.smithi042.stdout:4591 read 0xa536069 thru 0xa5398f2 (0x388a bytes) 2024-09-16T13:00:20.801 INFO:teuthology.orchestra.run.smithi042.stdout:4592 trunc from 0xba99bbe to 0x263505a 2024-09-16T13:00:20.806 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:20.799+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:20.814 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:20.814 INFO:teuthology.orchestra.run.smithi042.stdout:4594 writesame 0x1282bfe thru 0x12927bd data_size 0x98(0xfbc0 bytes) 2024-09-16T13:00:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:4596 write 0x92b61e3 thru 0x92b8771 (0x258f bytes) 2024-09-16T13:00:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:4597 read 0x2b1f14f thru 0x2b2bde2 (0xcc94 bytes) 2024-09-16T13:00:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:4598 write 0x800b762 thru 0x800e0bc (0x295b bytes) 2024-09-16T13:00:20.841 INFO:teuthology.orchestra.run.smithi042.stdout:4599 writesame 0x45484ca thru 0x4549fc1 data_size 0x4(0x1af8 bytes) 2024-09-16T13:00:20.844 INFO:teuthology.orchestra.run.smithi042.stdout:4600 punch from 0x81e942a to 0x81eeacd, (0x56a3 bytes) 2024-09-16T13:00:20.844 INFO:teuthology.orchestra.run.smithi042.stdout:4601 punch from 0xf8c694 to 0xf98312, (0xbc7e bytes) 2024-09-16T13:00:20.844 INFO:teuthology.orchestra.run.smithi042.stdout:4611 write 0xb54064 thru 0xb5d361 (0x92fe bytes) 2024-09-16T13:00:20.846 INFO:teuthology.orchestra.run.smithi042.stdout:4612 read 0x7b7ea8c thru 0x7b817e4 (0x2d59 bytes) 2024-09-16T13:00:20.847 INFO:teuthology.orchestra.run.smithi042.stdout:4613 clone 26 order 19 su 4096 sc 11 2024-09-16T13:00:20.862 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:20.863 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:20.872 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:20.914 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:20.907+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:21.008 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:21.002+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:21.017 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:21.017 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:21.026 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:21.109 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:21.099+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:21.127 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:21.127 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:21.136 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:21.210 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:21.206+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:21.235 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:21.235 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:21.244 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:21.311 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:21.303+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:21.346 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:21.346 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:21.355 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:21.397 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone25 from 0x92b8772 (overlap 0x4950b9) to 0x15fe9d 2024-09-16T13:00:21.411 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:21.406+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:21.459 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:21.459 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:21.468 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:21.512 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:21.503+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:21.515 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-16T13:00:21.613 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:21.606+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:21.615 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:21.615 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:21.624 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:21.713 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:21.707+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:21.719 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:21.719 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:21.727 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:21.815 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:21.807+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:21.818 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:21.819 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:21.828 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:21.916 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:21.910+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:21.922 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:21.922 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:21.931 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:21.955 INFO:teuthology.orchestra.run.smithi042.stdout:4614 writesame 0xb477041 thru 0xb483947 data_size 0x1(0xc907 bytes) 2024-09-16T13:00:21.955 INFO:teuthology.orchestra.run.smithi042.stdout:4616 punch from 0x780d85f to 0x7818681, (0xae22 bytes) 2024-09-16T13:00:21.955 INFO:teuthology.orchestra.run.smithi042.stdout:4618 read 0x7e85e2a thru 0x7e93974 (0xdb4b bytes) 2024-09-16T13:00:21.963 INFO:teuthology.orchestra.run.smithi042.stdout:4619 trunc from 0xb483948 to 0xa57bdac 2024-09-16T13:00:21.976 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:21.976 INFO:teuthology.orchestra.run.smithi042.stdout:4621 writesame 0x652daa3 thru 0x653bde4 data_size 0x13(0xe342 bytes) 2024-09-16T13:00:22.017 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:22.007+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:22.078 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:22.079 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:22.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:22.091 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:22.091 INFO:teuthology.orchestra.run.smithi042.stdout:4623 writesame 0x536acbd thru 0x5375e57 data_size 0x1(0xb19b bytes) 2024-09-16T13:00:22.117 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:22.110+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:22.161 INFO:teuthology.orchestra.run.smithi042.stdout:4624 punch from 0x1de0e97 to 0x1ded9d2, (0xcb3b bytes) 2024-09-16T13:00:22.169 INFO:teuthology.orchestra.run.smithi042.stdout:4625 write 0xac027bb thru 0xac06daa (0x45f0 bytes) 2024-09-16T13:00:22.178 INFO:teuthology.orchestra.run.smithi042.stdout:4626 write 0x3d06595 thru 0x3d10a7a (0xa4e6 bytes) 2024-09-16T13:00:22.218 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:22.211+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:22.242 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:22.242 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:22.255 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:22.280 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:22.280 INFO:teuthology.orchestra.run.smithi042.stdout:4629 writesame 0x17dcd22 thru 0x17dd7bf data_size 0x3(0xa9e bytes) 2024-09-16T13:00:22.297 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T13:00:22.305 INFO:teuthology.orchestra.run.smithi042.stdout:4631 write 0x12ac8c1 thru 0x12b2cda (0x641a bytes) 2024-09-16T13:00:22.319 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:22.314+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:22.369 INFO:teuthology.orchestra.run.smithi042.stdout:4632 punch from 0x3fe340c to 0x3ff15b5, (0xe1a9 bytes) 2024-09-16T13:00:22.383 INFO:teuthology.orchestra.run.smithi042.stdout:4633 write 0xe5378ff thru 0xe53fdcd (0x84cf bytes) 2024-09-16T13:00:22.402 INFO:teuthology.orchestra.run.smithi042.stdout:4634 write 0x82a6b93 thru 0x82b4ebd (0xe32b bytes) 2024-09-16T13:00:22.420 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:22.414+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:22.431 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:22.431 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:22.445 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:22.499 INFO:teuthology.orchestra.run.smithi042.stdout:4635 writesame 0xbf93778 thru 0xbf9f2c5 data_size 0x89(0xbb4e bytes) 2024-09-16T13:00:22.517 INFO:teuthology.orchestra.run.smithi042.stdout:4636 read 0xaa7b612 thru 0xaa87029 (0xba18 bytes) 2024-09-16T13:00:22.518 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:22.518 INFO:teuthology.orchestra.run.smithi042.stdout:4638 punch from 0xde81047 to 0xde882ad, (0x7266 bytes) 2024-09-16T13:00:22.521 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:22.515+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:22.522 INFO:teuthology.orchestra.run.smithi042.stdout:4639 trunc from 0xe53fdce to 0xbe98872 2024-09-16T13:00:22.540 INFO:teuthology.orchestra.run.smithi042.stdout:4643 write 0x86b0b2b thru 0x86bbba1 (0xb077 bytes) 2024-09-16T13:00:22.579 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:22.579 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:22.592 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:22.623 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:22.615+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:22.649 INFO:teuthology.orchestra.run.smithi042.stdout:4644 write 0xcce1749 thru 0xcce5d03 (0x45bb bytes) 2024-09-16T13:00:22.652 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:22.659 INFO:teuthology.orchestra.run.smithi042.stdout:4645 clone 27 order 19 su 2048 sc 4 2024-09-16T13:00:22.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T13:00:22.694+0000 7f9550397300 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T13:00:22.724 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:22.715+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:22.728 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:22.728 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:22.737 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:22.780 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:22.799 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:22.822 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:22.825 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:22.818+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:22.864 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:22.878 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:22.878 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:22.878 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:22.878 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:22.878 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:22.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:22.887 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:22.909 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:22.909 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:22.909 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:22.913 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:22.914 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:22.926 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:22.922+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54893 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.678970+0000", 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3253702169999997, 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056907577000000001, 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54893 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678970+0000", 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678970+0000", 2024-09-16T13:00:23.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678970+0000", 2024-09-16T13:00:23.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.27e-07 2024-09-16T13:00:23.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678976+0000", 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8710000000000002e-06 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678984+0000", 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5220000000000007e-06 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678988+0000", 2024-09-16T13:00:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9149999999999998e-06 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.735622+0000", 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056633731 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.735717+0000", 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4658000000000001e-05 2024-09-16T13:00:23.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.735877+0000", 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016045299999999999 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54894 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.678993+0000", 2024-09-16T13:00:23.015 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3253471339999994, 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057108937999999998, 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54894 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678993+0000", 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678993+0000", 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678993+0000", 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.678995+0000", 2024-09-16T13:00:23.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:00:23.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679004+0000", 2024-09-16T13:00:23.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9250000000000001e-06 2024-09-16T13:00:23.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679008+0000", 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8510000000000001e-06 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.735926+0000", 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056917945999999997 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736046+0000", 2024-09-16T13:00:23.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120673 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736102+0000", 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.516e-05 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54895 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 831488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.020 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679012+0000", 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3253276580000009, 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057255952999999998, 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54895 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679012+0000", 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679012+0000", 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679013+0000", 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2800000000000002e-07 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679015+0000", 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679023+0000", 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.799e-06 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679027+0000", 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6789999999999998e-06 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736130+0000", 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057102922 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736192+0000", 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2439000000000003e-05 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736268+0000", 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5717999999999996e-05 2024-09-16T13:00:23.024 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54896 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679031+0000", 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3253085949999992, 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057360204999999997, 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.025 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54896 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679031+0000", 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679031+0000", 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679032+0000", 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.39e-07 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679034+0000", 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0389999999999999e-06 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679042+0000", 2024-09-16T13:00:23.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4440000000000008e-06 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679046+0000", 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.721e-06 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736301+0000", 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057254699999999999 2024-09-16T13:00:23.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736339+0000", 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8424999999999999e-05 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736391+0000", 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2437e-05 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.032 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54897 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 897024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.032 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679050+0000", 2024-09-16T13:00:23.032 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3252898779999995, 2024-09-16T13:00:23.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057471563000000003, 2024-09-16T13:00:23.032 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54897 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679050+0000", 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679050+0000", 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679050+0000", 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679052+0000", 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0980000000000001e-06 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679061+0000", 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0330000000000008e-06 2024-09-16T13:00:23.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679064+0000", 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8160000000000004e-06 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736415+0000", 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05735109 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736462+0000", 2024-09-16T13:00:23.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.702e-05 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736522+0000", 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9094e-05 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54898 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679069+0000", 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3252713239999991, 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057588469000000003, 2024-09-16T13:00:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54898 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679069+0000", 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679069+0000", 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679069+0000", 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679071+0000", 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0559999999999999e-06 2024-09-16T13:00:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679077+0000", 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.964e-06 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679081+0000", 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7500000000000001e-06 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736551+0000", 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057470585999999997 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736596+0000", 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4725000000000003e-05 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736657+0000", 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0956999999999999e-05 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.040 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54899 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679084+0000", 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3252553480000007, 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057724701000000003, 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.041 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54899 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679084+0000", 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679084+0000", 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679085+0000", 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6199999999999999e-07 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679087+0000", 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8419999999999999e-06 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679095+0000", 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0919999999999998e-06 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679102+0000", 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3540000000000002e-06 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736687+0000", 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057585166 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736737+0000", 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9872e-05 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736809+0000", 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2013000000000003e-05 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54900 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.045 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679107+0000", 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3252333350000001, 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057834479000000001, 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54900 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679107+0000", 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679107+0000", 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679107+0000", 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6600000000000002e-07 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679109+0000", 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1449999999999998e-06 2024-09-16T13:00:23.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679117+0000", 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1440000000000003e-06 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679121+0000", 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.653e-06 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736842+0000", 2024-09-16T13:00:23.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057721040000000001 2024-09-16T13:00:23.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736888+0000", 2024-09-16T13:00:23.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6581e-05 2024-09-16T13:00:23.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736941+0000", 2024-09-16T13:00:23.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2549999999999997e-05 2024-09-16T13:00:23.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.050 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.050 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54901 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.050 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679125+0000", 2024-09-16T13:00:23.050 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3252149109999998, 2024-09-16T13:00:23.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057944962000000003, 2024-09-16T13:00:23.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54901 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679125+0000", 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679125+0000", 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679125+0000", 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679127+0000", 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.013e-06 2024-09-16T13:00:23.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679136+0000", 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679139+0000", 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.659e-06 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.736973+0000", 2024-09-16T13:00:23.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057833415999999999 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737017+0000", 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4427e-05 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737070+0000", 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2855000000000001e-05 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54902 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679143+0000", 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3251965099999996, 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058058212999999997, 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54902 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.055 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679143+0000", 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679143+0000", 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679144+0000", 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679146+0000", 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9649999999999998e-06 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679154+0000", 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2239999999999995e-06 2024-09-16T13:00:23.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679157+0000", 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4929999999999998e-06 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737101+0000", 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057943698000000002 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737149+0000", 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7453999999999997e-05 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737202+0000", 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2994000000000001e-05 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54903 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.059 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679161+0000", 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3251784830000002, 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145532, 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54903 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679161+0000", 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679161+0000", 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679162+0000", 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5499999999999998e-07 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679164+0000", 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0849999999999999e-06 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679170+0000", 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679173+0000", 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6729999999999998e-06 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737217+0000", 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058043933999999998 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737253+0000", 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6056000000000002e-05 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737307+0000", 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3440000000000003e-05 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.063 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54904 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1126400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679177+0000", 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3251624020000001, 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058235595000000001, 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54904 2024-09-16T13:00:23.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679177+0000", 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679177+0000", 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679178+0000", 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679180+0000", 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679188+0000", 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.585e-06 2024-09-16T13:00:23.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679192+0000", 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8920000000000002e-06 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737321+0000", 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058129382 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737356+0000", 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4851999999999998e-05 2024-09-16T13:00:23.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737413+0000", 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6716999999999999e-05 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54905 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679196+0000", 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3251439999999999, 2024-09-16T13:00:23.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058328532000000002, 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54905 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679196+0000", 2024-09-16T13:00:23.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679196+0000", 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679196+0000", 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679217+0000", 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0645e-05 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679228+0000", 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0728e-05 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679232+0000", 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7990000000000001e-06 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737428+0000", 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058195445999999998 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737471+0000", 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3585999999999999e-05 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737524+0000", 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2932000000000001e-05 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54906 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1191936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679237+0000", 2024-09-16T13:00:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3251024059999992, 2024-09-16T13:00:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442112999999997, 2024-09-16T13:00:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54906 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679237+0000", 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679237+0000", 2024-09-16T13:00:23.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679238+0000", 2024-09-16T13:00:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-07 2024-09-16T13:00:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679240+0000", 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2369999999999999e-06 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679249+0000", 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999999e-06 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679252+0000", 2024-09-16T13:00:23.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6710000000000001e-06 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737539+0000", 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058286642 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737586+0000", 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6900999999999997e-05 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737680+0000", 2024-09-16T13:00:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3604999999999996e-05 2024-09-16T13:00:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54907 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1224704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679257+0000", 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3250825880000008, 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058610582000000001, 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54907 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.079 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679257+0000", 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679257+0000", 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679258+0000", 2024-09-16T13:00:23.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679260+0000", 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9640000000000002e-06 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679268+0000", 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6039999999999993e-06 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679272+0000", 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8829999999999999e-06 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737712+0000", 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439921999999998 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737797+0000", 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4739000000000003e-05 2024-09-16T13:00:23.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737868+0000", 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1069999999999995e-05 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54908 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679276+0000", 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3250634390000009, 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058697892000000002, 2024-09-16T13:00:23.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54908 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679276+0000", 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679276+0000", 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679277+0000", 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7300000000000002e-07 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679279+0000", 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679285+0000", 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7679999999999997e-06 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679288+0000", 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.568e-06 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737884+0000", 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058596033999999998 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737923+0000", 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8659999999999999e-05 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737974+0000", 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1520999999999997e-05 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54909 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679292+0000", 2024-09-16T13:00:23.087 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3250481349999994, 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058833353999999997, 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54909 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679292+0000", 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679292+0000", 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679292+0000", 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679294+0000", 2024-09-16T13:00:23.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.888e-06 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679302+0000", 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3559999999999993e-06 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679306+0000", 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3500000000000001e-06 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.737989+0000", 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058683524000000001 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738029+0000", 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9854999999999997e-05 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738125+0000", 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6003999999999998e-05 2024-09-16T13:00:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54910 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1323008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679310+0000", 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3250301340000004, 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059166712000000003, 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54910 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679310+0000", 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679310+0000", 2024-09-16T13:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679310+0000", 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.34e-07 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679312+0000", 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0219999999999999e-06 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679321+0000", 2024-09-16T13:00:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3629999999999994e-06 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679324+0000", 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.738e-06 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738187+0000", 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058862652000000001 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738318+0000", 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013088299999999999 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738476+0000", 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015862 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54911 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1355776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679328+0000", 2024-09-16T13:00:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3250113750000008, 2024-09-16T13:00:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059496683000000002, 2024-09-16T13:00:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54911 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679328+0000", 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679328+0000", 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679329+0000", 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679331+0000", 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9069999999999999e-06 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679339+0000", 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7469999999999994e-06 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679343+0000", 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.749e-06 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738522+0000", 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059178423000000001 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738658+0000", 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000136766 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.738825+0000", 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016672099999999999 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54912 4.12 4:4a54b4f7:::rbd_data.11b58f0917ce.0000000000000001:1a [read 1388544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.679358+0000", 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3249819330000001, 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060353395999999997, 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54912 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679358+0000", 2024-09-16T13:00:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679358+0000", 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679358+0000", 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679360+0000", 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.108e-06 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679369+0000", 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6850000000000007e-06 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.679373+0000", 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.895e-06 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.739469+0000", 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060096019000000001 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.739561+0000", 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1526000000000004e-05 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.739711+0000", 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015069899999999999 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.106 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:23.107 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:23.108 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:23.102+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:23.208 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:23.202+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:23.216 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:23.216 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:23.226 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:23.295 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T13:00:23.309 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:23.302+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:23.337 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:23.337 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:23.338 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:23.338 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:23.338 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:23.348 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:23.409 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:23.403+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:23.415 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone26 from 0xcce5d04 (overlap 0x92b8772) to 0x5808477 2024-09-16T13:00:23.471 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:23.471 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:23.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:23.471 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54223 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.810837867, 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028808500000000001, 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54223 2024-09-16T13:00:23.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655799+0000", 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655801+0000", 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6579999999999998e-06 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655811+0000", 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5070000000000006e-06 2024-09-16T13:00:23.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655816+0000", 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8960000000000002e-06 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684450+0000", 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028634057000000001 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684521+0000", 2024-09-16T13:00:23.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1453999999999997e-05 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684607+0000", 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5447999999999999e-05 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54224 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1220608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:23.476 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8108150649999999, 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028979718000000002, 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54224 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655822+0000", 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2600000000000002e-07 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655824+0000", 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.593e-06 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655833+0000", 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2099999999999999e-06 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655838+0000", 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5650000000000003e-06 2024-09-16T13:00:23.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684637+0000", 2024-09-16T13:00:23.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028798550999999999 2024-09-16T13:00:23.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684715+0000", 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8242000000000001e-05 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684801+0000", 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6031000000000004e-05 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54225 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:23.481 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8107929479999996, 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029147567999999999, 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54225 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655844+0000", 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.06e-07 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655846+0000", 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5059999999999998e-06 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655856+0000", 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.825e-06 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655861+0000", 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6990000000000002e-06 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684830+0000", 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028969385 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684907+0000", 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6800999999999999e-05 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684991+0000", 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3845999999999996e-05 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54226 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.485 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8107700080000004, 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029452944000000002, 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54226 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655867+0000", 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8599999999999998e-07 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655869+0000", 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5849999999999998e-06 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655878+0000", 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2620000000000004e-06 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655883+0000", 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7350000000000004e-06 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685017+0000", 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029134083000000002 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685096+0000", 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999994e-05 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685319+0000", 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022352299999999999 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.489 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54227 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1318912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8107475209999997, 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029655159, 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54227 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.490 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5700000000000002e-07 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655892+0000", 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.486e-06 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655903+0000", 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1e-05 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655907+0000", 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.831e-06 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685347+0000", 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029439844 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685416+0000", 2024-09-16T13:00:23.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8561999999999998e-05 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685544+0000", 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012787900000000001 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54228 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:23.494 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8107231759999998, 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029867089999999999, 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54228 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2499999999999995e-07 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655916+0000", 2024-09-16T13:00:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9170000000000002e-06 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655926+0000", 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7769999999999997e-06 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655931+0000", 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8470000000000003e-06 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685571+0000", 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029640061999999998 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685642+0000", 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1327000000000003e-05 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685780+0000", 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013763499999999999 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.498 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54229 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8106994689999993, 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030082340999999999, 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54229 2024-09-16T13:00:23.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1500000000000005e-07 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655940+0000", 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5840000000000002e-06 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655949+0000", 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9979999999999992e-06 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655954+0000", 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.746e-06 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685808+0000", 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029854776999999999 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685879+0000", 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0593000000000007e-05 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.686019+0000", 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000140128 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54832 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 192512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7640463040000007, 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.032379022, 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54832 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4099999999999999e-07 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702592+0000", 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.995e-06 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702597+0000", 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2120000000000002e-06 2024-09-16T13:00:23.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702600+0000", 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9919999999999999e-06 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.727494+0000", 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024893141000000001 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.727564+0000", 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0761999999999998e-05 2024-09-16T13:00:23.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734969+0000", 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0074044790000000003 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54858 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1044480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.762727924, 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028959381999999999, 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54858 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:23.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703909+0000", 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7599999999999997e-07 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703911+0000", 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4789999999999999e-06 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703917+0000", 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2389999999999999e-06 2024-09-16T13:00:23.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703920+0000", 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.021e-06 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732697+0000", 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028776665 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732778+0000", 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1018000000000003e-05 2024-09-16T13:00:23.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732868+0000", 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9483999999999999e-05 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54859 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1077248~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7627104889999998, 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029138931999999999, 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54859 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9400000000000001e-07 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703928+0000", 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1179999999999999e-06 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703934+0000", 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.84e-06 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703938+0000", 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5219999999999999e-06 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732893+0000", 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028955005999999998 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732974+0000", 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1569000000000005e-05 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733065+0000", 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0482999999999994e-05 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54860 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1110016~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7626944099999999, 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029318429999999999, 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54860 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:23.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703944+0000", 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9209999999999999e-06 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704020+0000", 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6217999999999994e-05 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704027+0000", 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3559999999999998e-06 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733091+0000", 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029064349999999999 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733167+0000", 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6058999999999998e-05 2024-09-16T13:00:23.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733260+0000", 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3110999999999997e-05 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54861 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7626032190000007, 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029431589000000001, 2024-09-16T13:00:23.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54861 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704034+0000", 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6199999999999997e-07 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704037+0000", 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9100000000000001e-06 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704045+0000", 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2150000000000001e-06 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704048+0000", 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6380000000000002e-06 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733285+0000", 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029236999999999999 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733365+0000", 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9461000000000004e-05 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733465+0000", 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9603000000000002e-05 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.524 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54862 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1175552~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7625835470000002, 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029595202000000001, 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54862 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:23.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3300000000000002e-07 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704055+0000", 2024-09-16T13:00:23.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0940000000000002e-06 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704061+0000", 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.835e-06 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704064+0000", 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.948e-06 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733485+0000", 2024-09-16T13:00:23.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029420565999999999 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733552+0000", 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7409999999999993e-05 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733648+0000", 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5816000000000006e-05 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54863 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1208320~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7625687489999997, 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029759681, 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54863 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704068+0000", 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704076+0000", 2024-09-16T13:00:23.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0299999999999994e-06 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704082+0000", 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.755e-06 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704085+0000", 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1159999999999999e-06 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733670+0000", 2024-09-16T13:00:23.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029585163000000001 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733742+0000", 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2069000000000004e-05 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733827+0000", 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5161000000000007e-05 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54864 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7625476249999998, 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927993, 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54864 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.533 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:23.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3599999999999999e-07 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704091+0000", 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-06 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704096+0000", 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0950000000000003e-06 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704099+0000", 2024-09-16T13:00:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8329999999999998e-06 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733853+0000", 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029754368999999999 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733931+0000", 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7956999999999999e-05 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734017+0000", 2024-09-16T13:00:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5272999999999995e-05 2024-09-16T13:00:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54865 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1273856~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.537 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7625336919999999, 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030108264999999999, 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54865 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704103+0000", 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704104+0000", 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.716e-06 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704110+0000", 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4410000000000001e-06 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704113+0000", 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1839999999999999e-06 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734040+0000", 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927013999999998 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734122+0000", 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1481000000000001e-05 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734211+0000", 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9140000000000004e-05 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54866 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1306624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7625192809999994, 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030274079999999998, 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54866 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704119+0000", 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9870000000000002e-06 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704125+0000", 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3410000000000002e-06 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704133+0000", 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734225+0000", 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030092026000000001 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734306+0000", 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0634000000000001e-05 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734391+0000", 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5082999999999998e-05 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54867 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7624992079999995, 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030392758999999998, 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54867 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.546 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704139+0000", 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.99e-06 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704145+0000", 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704148+0000", 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.078e-06 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734412+0000", 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030264287000000001 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734469+0000", 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6925999999999997e-05 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734530+0000", 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0381999999999999e-05 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54868 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1372160~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7624845130000004, 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030530140000000001, 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54868 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704245+0000", 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3037999999999999e-05 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704256+0000", 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0718e-05 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704261+0000", 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734544+0000", 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030283929000000001 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734595+0000", 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0356999999999999e-05 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734682+0000", 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6938e-05 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.554 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54869 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1404928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7623702730000002, 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030584762000000001, 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54869 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:23.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704267+0000", 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4700000000000001e-07 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704269+0000", 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1390000000000002e-06 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704275+0000", 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9449999999999998e-06 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704278+0000", 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1429999999999998e-06 2024-09-16T13:00:23.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734705+0000", 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030427551000000001 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734778+0000", 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2502000000000005e-05 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:23.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734851+0000", 2024-09-16T13:00:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2835000000000003e-05 2024-09-16T13:00:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.559 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:23.560 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:23.561 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:23.551+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:23.662 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:23.651+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80650 4.1b 4.c6ce51db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.742894+0000", 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0055284510000000002, 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055531030000000002, 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80650 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742894+0000", 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742894+0000", 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742895+0000", 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742908+0000", 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3366e-05 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742925+0000", 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6866000000000001e-05 2024-09-16T13:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742930+0000", 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0350000000000001e-06 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80651 4.1b 4.c6ce51db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.742936+0000", 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0054870320000000002, 2024-09-16T13:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0055776949999999997, 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80651 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742936+0000", 2024-09-16T13:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742936+0000", 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742936+0000", 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1200000000000003e-07 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742939+0000", 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0149999999999999e-06 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742946+0000", 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3670000000000002e-06 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742949+0000", 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7519999999999998e-06 2024-09-16T13:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80652 4.1b 4.c6ce51db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.742954+0000", 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0054684850000000004, 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005616411, 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80652 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742954+0000", 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742954+0000", 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742955+0000", 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742957+0000", 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.979e-06 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742963+0000", 2024-09-16T13:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9229999999999999e-06 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742967+0000", 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7840000000000001e-06 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80653 4.1b 4.c6ce51db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.742986+0000", 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0054371630000000001, 2024-09-16T13:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056427209999999998, 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80653 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742986+0000", 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742986+0000", 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742986+0000", 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.32e-07 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742989+0000", 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5160000000000001e-06 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742995+0000", 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8190000000000003e-06 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743000+0000", 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2200000000000003e-06 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80654 4.1b 4.c6ce51db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.743004+0000", 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0054187899999999997, 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056804029999999997, 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80654 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743004+0000", 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743004+0000", 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743005+0000", 2024-09-16T13:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3200000000000005e-07 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743007+0000", 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.052e-06 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743022+0000", 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5321000000000001e-05 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743032+0000", 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8330000000000006e-06 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80655 4.1b 4.c6ce51db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.743037+0000", 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0053862110000000001, 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0057052420000000001, 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80655 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743037+0000", 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743037+0000", 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743037+0000", 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.27e-07 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743039+0000", 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3470000000000001e-06 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743046+0000", 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1630000000000001e-06 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.743049+0000", 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8319999999999999e-06 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 12939264~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.730368+0000", 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.018054389000000001, 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018434327, 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80512 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.730368+0000", 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.730368+0000", 2024-09-16T13:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.730369+0000", 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.730371+0000", 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.046e-06 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.730377+0000", 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8069999999999996e-06 2024-09-16T13:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.730380+0000", 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6480000000000001e-06 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.746559+0000", 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016178896000000002 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.746642+0000", 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2404000000000006e-05 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80716 4.11 4:8bbc91c6:::rbd_data.11b5c2ee3542.0000000000000007:1d [read 12640256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.746654+0000", 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0017684, 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022184259999999999, 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80716 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.746654+0000", 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.746654+0000", 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.746655+0000", 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8200000000000001e-07 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.746657+0000", 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.086e-06 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.746663+0000", 2024-09-16T13:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3439999999999997e-06 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.746667+0000", 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7740000000000002e-06 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748837+0000", 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021695899999999999 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80644 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 12853248~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.742608+0000", 2024-09-16T13:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0058144360000000001, 2024-09-16T13:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063298010000000004, 2024-09-16T13:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80644 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742608+0000", 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742608+0000", 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742609+0000", 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0640000000000001e-06 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742615+0000", 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1229999999999999e-06 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742776+0000", 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000161762 2024-09-16T13:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742784+0000", 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1389999999999998e-06 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748832+0000", 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060481040000000003 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748878+0000", 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6823000000000002e-05 2024-09-16T13:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80717 4.11 4.63893dd1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.748813+0000", 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9996095, 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000199493, 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80717 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748813+0000", 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748813+0000", 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748814+0000", 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2399999999999996e-07 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748818+0000", 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748826+0000", 2024-09-16T13:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9570000000000002e-06 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748831+0000", 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3349999999999997e-06 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.749021+0000", 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019018600000000001 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80645 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 12898304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.742792+0000", 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0056310450000000003, 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062903730000000001, 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80645 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742792+0000", 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742792+0000", 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742793+0000", 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5099999999999999e-07 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742796+0000", 2024-09-16T13:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6830000000000001e-06 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742804+0000", 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5700000000000004e-06 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742808+0000", 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0729999999999998e-06 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.749008+0000", 2024-09-16T13:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062004449999999997 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.749074+0000", 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5661999999999996e-05 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80718 4.11 4.63893dd1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.748837+0000", 2024-09-16T13:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9995861, 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031547699999999999, 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80718 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748837+0000", 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748837+0000", 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748837+0000", 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1500000000000005e-07 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748839+0000", 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1390000000000002e-06 2024-09-16T13:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748845+0000", 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.314e-06 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748849+0000", 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9689999999999996e-06 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80646 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 12943360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.742813+0000", 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0056102269999999997, 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0064047610000000001, 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80646 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742813+0000", 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742813+0000", 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742813+0000", 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742815+0000", 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742821+0000", 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1199999999999999e-06 2024-09-16T13:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742825+0000", 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.907e-06 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.749169+0000", 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063433719999999999 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80719 4.11 4.63893dd1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.748854+0000", 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9995689, 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000425635, 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80719 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748854+0000", 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748854+0000", 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748854+0000", 2024-09-16T13:00:23.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8200000000000001e-07 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748856+0000", 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1289999999999999e-06 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748862+0000", 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7119999999999997e-06 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748866+0000", 2024-09-16T13:00:23.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8139999999999998e-06 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80647 4.1b 4.c6ce51db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.742830+0000", 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0055932009999999999, 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0065047969999999997, 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80647 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742830+0000", 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742830+0000", 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742830+0000", 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742833+0000", 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4229999999999999e-06 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742839+0000", 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0599999999999996e-06 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742842+0000", 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8469999999999998e-06 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80720 4.11 4.63893dd1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.748870+0000", 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9995532, 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052425300000000002, 2024-09-16T13:00:23.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80720 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748870+0000", 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748870+0000", 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748870+0000", 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748872+0000", 2024-09-16T13:00:23.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0109999999999999e-06 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748878+0000", 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8449999999999999e-06 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748882+0000", 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8020000000000002e-06 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80648 4.1b 4.c6ce51db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.742846+0000", 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0055765099999999998, 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066085939999999998, 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80648 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742846+0000", 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742846+0000", 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742847+0000", 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.01e-07 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742849+0000", 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3439999999999999e-06 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742855+0000", 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8109999999999999e-06 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742859+0000", 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7069999999999998e-06 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80721 4.11 4.63893dd1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.748886+0000", 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.999537, 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000628325, 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80721 2024-09-16T13:00:23.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748886+0000", 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748886+0000", 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748886+0000", 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:00:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748888+0000", 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9609999999999999e-06 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748894+0000", 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.835e-06 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748899+0000", 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7679999999999999e-06 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80649 4.1b 4.c6ce51db (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.742863+0000", 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0055602730000000001, 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067258819999999999, 2024-09-16T13:00:23.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80649 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742863+0000", 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742863+0000", 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742863+0000", 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0400000000000002e-07 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742865+0000", 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9549999999999999e-06 2024-09-16T13:00:23.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742871+0000", 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.742874+0000", 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6030000000000001e-06 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:80722 4.11 4.63893dd1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:23.748903+0000", 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9995198, 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00074261900000000003, 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 80722 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:23.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748903+0000", 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748903+0000", 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748903+0000", 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6100000000000002e-07 2024-09-16T13:00:23.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748910+0000", 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0360000000000001e-06 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748918+0000", 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5979999999999999e-06 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:23.748922+0000", 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.737e-06 2024-09-16T13:00:23.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.823 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:23.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:23.823 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:00:23.823 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 20 2024-09-16T13:00:23.823 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:23.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:23.825 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:23.815+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:23.925 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:23.918+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:23.952 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:23.952 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:23.952 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:23.952 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:23.952 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:23.965 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:23.999 INFO:teuthology.orchestra.run.smithi136.stdout:4.e 2024-09-16T13:00:24.000 INFO:teuthology.orchestra.run.smithi136.stdout:4.c 2024-09-16T13:00:24.000 INFO:teuthology.orchestra.run.smithi136.stdout:4.1a 2024-09-16T13:00:24.000 INFO:teuthology.orchestra.run.smithi136.stdout:4.1 2024-09-16T13:00:24.000 INFO:teuthology.orchestra.run.smithi136.stdout:4.18 2024-09-16T13:00:24.000 INFO:teuthology.orchestra.run.smithi136.stdout:4.5 2024-09-16T13:00:24.000 INFO:teuthology.orchestra.run.smithi136.stdout:4.13 2024-09-16T13:00:24.000 INFO:teuthology.orchestra.run.smithi136.stdout:4.d 2024-09-16T13:00:24.000 INFO:teuthology.orchestra.run.smithi136.stdout:4.1b 2024-09-16T13:00:24.001 INFO:teuthology.orchestra.run.smithi136.stdout:3.3 2024-09-16T13:00:24.001 INFO:teuthology.orchestra.run.smithi136.stdout:4.9 2024-09-16T13:00:24.001 INFO:teuthology.orchestra.run.smithi136.stdout:3.1 2024-09-16T13:00:24.025 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:24.015+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47613 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3571712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707195752000001, 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054671112000000001, 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47613 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357528+0000", 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5699999999999999e-07 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357529+0000", 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1459999999999999e-06 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357532+0000", 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357534+0000", 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9089999999999998e-06 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412120+0000", 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054585897000000001 2024-09-16T13:00:24.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412159+0000", 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8967000000000001e-05 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412199+0000", 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0068999999999999e-05 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47614 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3588096~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357536+0000", 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707187055, 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054770905000000002, 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47614 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.074 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357536+0000", 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357536+0000", 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357537+0000", 2024-09-16T13:00:24.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357538+0000", 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0950000000000001e-06 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357541+0000", 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9979999999999999e-06 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357543+0000", 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9539999999999998e-06 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412227+0000", 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054684501000000003 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412266+0000", 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8764999999999999e-05 2024-09-16T13:00:24.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412307+0000", 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1341000000000003e-05 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47615 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3604480~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707178750000001, 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054886719, 2024-09-16T13:00:24.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47615 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357545+0000", 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.41e-07 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357546+0000", 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.099e-06 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357549+0000", 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8820000000000001e-06 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357551+0000", 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.852e-06 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412335+0000", 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054784294999999997 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412392+0000", 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6938000000000003e-05 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412431+0000", 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T13:00:24.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47616 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3620864~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707170671, 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054998867, 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47616 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357553+0000", 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4900000000000002e-07 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357554+0000", 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0669999999999999e-06 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357557+0000", 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9079999999999999e-06 2024-09-16T13:00:24.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357559+0000", 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9580000000000001e-06 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412469+0000", 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.054909873999999997 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412512+0000", 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3430999999999998e-05 2024-09-16T13:00:24.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412552+0000", 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9379999999999999e-05 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47617 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3637248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707162507, 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055098616000000003, 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47617 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:24.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357561+0000", 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6e-07 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357562+0000", 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0809999999999999e-06 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357565+0000", 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357567+0000", 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412578+0000", 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055010816999999997 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412615+0000", 2024-09-16T13:00:24.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7629999999999997e-05 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412660+0000", 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4116999999999998e-05 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47618 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3653632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707153203000001, 2024-09-16T13:00:24.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055192228000000003, 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47618 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357570+0000", 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3099999999999999e-07 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357573+0000", 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9919999999999999e-06 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357577+0000", 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.208e-06 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357579+0000", 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412685+0000", 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055105981999999998 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412723+0000", 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8028999999999998e-05 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412762+0000", 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9221999999999998e-05 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47619 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3670016~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357581+0000", 2024-09-16T13:00:24.095 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707142055, 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055287718999999999, 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47619 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357581+0000", 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357581+0000", 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357582+0000", 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6100000000000002e-07 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357583+0000", 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.085e-06 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357586+0000", 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.931e-06 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357588+0000", 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412788+0000", 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055200013999999999 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412827+0000", 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9002999999999999e-05 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412869+0000", 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2546000000000002e-05 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.099 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47620 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3686400~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357590+0000", 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707133117, 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0553897, 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47620 2024-09-16T13:00:24.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357590+0000", 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357590+0000", 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357591+0000", 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357592+0000", 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.06e-06 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357595+0000", 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.999e-06 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357597+0000", 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9640000000000002e-06 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412900+0000", 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055303123000000003 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412941+0000", 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0907e-05 2024-09-16T13:00:24.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.412980+0000", 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9412000000000001e-05 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47621 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3702784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357598+0000", 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707125041999999, 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055490600000000001, 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47621 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357598+0000", 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357598+0000", 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357599+0000", 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5199999999999998e-07 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357600+0000", 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357603+0000", 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9560000000000002e-06 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357606+0000", 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1300000000000001e-06 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413012+0000", 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055406186000000003 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413051+0000", 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8686999999999997e-05 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413089+0000", 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8223000000000003e-05 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47622 4.1d 4:b96240c3:::rbd_data.11b52dde2e.0000000000000000:18 [read 3719168~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:24.108 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707115489, 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055581815, 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47622 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357608+0000", 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4699999999999998e-07 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357609+0000", 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7399999999999991e-07 2024-09-16T13:00:24.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357612+0000", 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9799999999999998e-06 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357615+0000", 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.269e-06 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413115+0000", 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055499522000000003 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413154+0000", 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8584999999999997e-05 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413190+0000", 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6238000000000001e-05 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.112 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47623 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357617+0000", 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707106033000001, 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055603054999999998, 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47623 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357617+0000", 2024-09-16T13:00:24.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357617+0000", 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357618+0000", 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6199999999999999e-07 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357619+0000", 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.032e-06 2024-09-16T13:00:24.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357622+0000", 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9679999999999998e-06 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357624+0000", 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413215+0000", 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055591451 2024-09-16T13:00:24.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413220+0000", 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4620000000000004e-06 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47624 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357625+0000", 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.70709802, 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055617605000000001, 2024-09-16T13:00:24.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47624 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357625+0000", 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357625+0000", 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357626+0000", 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4900000000000002e-07 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357627+0000", 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2139999999999999e-06 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357630+0000", 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9409999999999999e-06 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357633+0000", 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6960000000000001e-06 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413236+0000", 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055603303999999999 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413243+0000", 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2010000000000002e-06 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47625 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707087462, 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055628134000000003, 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47625 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357636+0000", 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357637+0000", 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5300000000000002e-07 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357640+0000", 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357642+0000", 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9130000000000001e-06 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413257+0000", 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055615255000000002 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413264+0000", 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.849e-06 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47626 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707079476000001, 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055641412000000001, 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47626 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357644+0000", 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5600000000000002e-07 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357645+0000", 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-06 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357648+0000", 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9859999999999999e-06 2024-09-16T13:00:24.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357650+0000", 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.922e-06 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413278+0000", 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055628089999999998 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413285+0000", 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1289999999999998e-06 2024-09-16T13:00:24.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47627 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707071304999999, 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055653995999999997, 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47627 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357652+0000", 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357654+0000", 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.212e-06 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357656+0000", 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.897e-06 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357658+0000", 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8920000000000001e-06 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413299+0000", 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055640991000000001 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413306+0000", 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47628 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357660+0000", 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707063113, 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055666457000000003, 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47628 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357660+0000", 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357660+0000", 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357661+0000", 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5499999999999999e-07 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357662+0000", 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.065e-06 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357665+0000", 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357668+0000", 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413320+0000", 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055652245000000003 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413327+0000", 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7989999999999996e-06 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47629 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357670+0000", 2024-09-16T13:00:24.135 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707053203999999, 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055676011999999997, 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47629 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357670+0000", 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357670+0000", 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357671+0000", 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4699999999999998e-07 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357672+0000", 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.015e-06 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357675+0000", 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0359999999999998e-06 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357677+0000", 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.481e-06 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413340+0000", 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662752000000003 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413346+0000", 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4810000000000003e-06 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47630 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:24.139 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707044440000001, 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055688423000000001, 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47630 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:24.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357679+0000", 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357680+0000", 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.11e-06 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357683+0000", 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1250000000000001e-06 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357686+0000", 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-06 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413360+0000", 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055674003 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413367+0000", 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9189999999999999e-06 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.143 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47631 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357688+0000", 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.707035934, 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055700309000000003, 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47631 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357688+0000", 2024-09-16T13:00:24.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357688+0000", 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357733+0000", 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5716000000000001e-05 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357740+0000", 2024-09-16T13:00:24.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3740000000000003e-06 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357748+0000", 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4919999999999993e-06 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357753+0000", 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1259999999999997e-06 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413381+0000", 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055627958999999998 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413388+0000", 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:47632 4.1d 4.c302469d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:10.357759+0000", 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.706964458, 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055652716999999997, 2024-09-16T13:00:24.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 47632 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357759+0000", 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.357759+0000", 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359561+0000", 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018016060000000001 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359564+0000", 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359577+0000", 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3087e-05 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.359582+0000", 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2549999999999999e-06 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413404+0000", 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.053822163999999999 2024-09-16T13:00:24.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:10.413412+0000", 2024-09-16T13:00:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0330000000000008e-06 2024-09-16T13:00:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:24.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:24.151 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:24.152 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:24.153 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:24.146+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:24.188 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-16T13:00:24.200 INFO:teuthology.orchestra.run.smithi042.stdout:4648 read 0x758d97f thru 0x759a582 (0xcc04 bytes) 2024-09-16T13:00:24.209 INFO:teuthology.orchestra.run.smithi042.stdout:4649 clone 28 order 24 su 262144 sc 3 2024-09-16T13:00:24.253 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:24.243+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:24.255 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:24.255 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:24.255 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:24.256 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:24.265 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:24.354 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:24.346+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:24.365 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:24.365 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:24.366 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:24.366 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:24.366 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:24.375 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:24.454 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:24.450+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:24.462 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone27 from 0xcce5d04 (overlap 0xcce5d04) to 0x147d61a 2024-09-16T13:00:24.473 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8419 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.065152288, 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040293592000000003, 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8419 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402894+0000", 2024-09-16T13:00:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402907+0000", 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402940+0000", 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3436000000000003e-05 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402953+0000", 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2840000000000001e-05 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403104+0000", 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150712 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403202+0000", 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8403e-05 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.411015+0000", 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078133079999999997 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443018+0000", 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032002348999999999 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443075+0000", 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7703e-05 2024-09-16T13:00:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443141+0000", 2024-09-16T13:00:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5705999999999997e-05 2024-09-16T13:00:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443185+0000", 2024-09-16T13:00:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4152e-05 2024-09-16T13:00:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.50191993, 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039945895000000002, 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48384 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966126+0000", 2024-09-16T13:00:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1110000000000002e-06 2024-09-16T13:00:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966134+0000", 2024-09-16T13:00:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9740000000000002e-06 2024-09-16T13:00:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966139+0000", 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4270000000000001e-06 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.005829+0000", 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039690007999999999 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.005928+0000", 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9184000000000004e-05 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006070+0000", 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014171100000000001 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501900299000001, 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040120731, 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48385 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.63e-07 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966146+0000", 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.018e-06 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966154+0000", 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9999999999999996e-06 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966169+0000", 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5395000000000001e-05 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006118+0000", 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039948637000000002 2024-09-16T13:00:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006181+0000", 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3001000000000002e-05 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006264+0000", 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3217e-05 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501869176, 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040378561, 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48386 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5199999999999996e-07 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966178+0000", 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.373e-06 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966186+0000", 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3259999999999996e-06 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966202+0000", 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6333000000000001e-05 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006311+0000", 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040108447999999998 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006411+0000", 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100141 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006553+0000", 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014218799999999999 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501835645, 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040537542000000003, 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48387 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966209+0000", 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4300000000000003e-07 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966211+0000", 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.418e-06 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966219+0000", 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.627e-06 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966223+0000", 2024-09-16T13:00:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3019999999999996e-06 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006601+0000", 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040377953000000001 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006665+0000", 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3592999999999999e-05 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006746+0000", 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1106000000000006e-05 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501816047, 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040793276000000003, 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.499 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48388 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5700000000000002e-07 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966230+0000", 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0499999999999999e-06 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966246+0000", 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5543e-05 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966250+0000", 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0840000000000002e-06 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006794+0000", 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040543691999999999 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006901+0000", 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107754 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007021+0000", 2024-09-16T13:00:24.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119596 2024-09-16T13:00:24.503 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.503 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.503 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.503 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.503 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501788647, 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040958857000000001, 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48389 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966256+0000", 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8100000000000003e-07 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966258+0000", 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3829999999999999e-06 2024-09-16T13:00:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966268+0000", 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8379999999999997e-06 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966272+0000", 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9940000000000002e-06 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007069+0000", 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040796897999999998 2024-09-16T13:00:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007132+0000", 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3084999999999997e-05 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007214+0000", 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2177999999999998e-05 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501766169, 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041219483000000001, 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48390 2024-09-16T13:00:24.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:24.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4700000000000002e-07 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966280+0000", 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3e-06 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966286+0000", 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.57e-06 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966288+0000", 2024-09-16T13:00:24.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4439999999999998e-06 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007262+0000", 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040973242 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007351+0000", 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9339999999999995e-05 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007497+0000", 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014614000000000001 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501744506, 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041394590000000002, 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.512 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48391 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966300+0000", 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966301+0000", 2024-09-16T13:00:24.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5510000000000001e-06 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966305+0000", 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1849999999999997e-06 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966310+0000", 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.211e-06 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007546+0000", 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041236341000000003 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007610+0000", 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3534999999999994e-05 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007694+0000", 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4407999999999996e-05 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.516 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48392 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 794624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501731852000001, 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041645095, 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48392 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.517 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3599999999999999e-07 2024-09-16T13:00:24.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966314+0000", 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7689999999999999e-06 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966322+0000", 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9340000000000006e-06 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966325+0000", 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5579999999999999e-06 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007742+0000", 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041417127999999998 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007840+0000", 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8199e-05 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007957+0000", 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011717099999999999 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48393 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501716655999999, 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041849738999999997, 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.521 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48393 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966328+0000", 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966340+0000", 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2604e-05 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966344+0000", 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1989999999999999e-06 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966348+0000", 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4039999999999999e-06 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008030+0000", 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041682036999999998 2024-09-16T13:00:24.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008093+0000", 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3511999999999998e-05 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008177+0000", 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3561999999999997e-05 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48394 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501693197, 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042095443000000003, 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48394 2024-09-16T13:00:24.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:24.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966352+0000", 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.406e-06 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966356+0000", 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:00:24.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966362+0000", 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2299999999999996e-06 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008232+0000", 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041869361000000001 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008320+0000", 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8492000000000006e-05 2024-09-16T13:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008446+0000", 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012591800000000001 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48395 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501677656, 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042265693999999999, 2024-09-16T13:00:24.530 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48395 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966367+0000", 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966368+0000", 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9420000000000002e-06 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966372+0000", 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9049999999999999e-06 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966375+0000", 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4329999999999998e-06 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008483+0000", 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042108244000000003 2024-09-16T13:00:24.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008546+0000", 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2973999999999997e-05 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008632+0000", 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.585e-05 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.534 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48396 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501666568999999, 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042505605000000002, 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48396 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:24.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966378+0000", 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966379+0000", 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.147e-06 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966382+0000", 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966385+0000", 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.43e-06 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008677+0000", 2024-09-16T13:00:24.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042292191 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008765+0000", 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8027000000000003e-05 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008883+0000", 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117817 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48397 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501656663, 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042627777999999998, 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48397 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966388+0000", 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966389+0000", 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.144e-06 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966392+0000", 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5939999999999998e-06 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966398+0000", 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3959999999999997e-06 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008904+0000", 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042505730999999998 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008953+0000", 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9786000000000001e-05 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009015+0000", 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1580000000000003e-05 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.542 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48398 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501643321, 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043115957000000003, 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.543 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48398 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2500000000000001e-07 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966402+0000", 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3120000000000001e-06 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966406+0000", 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.653e-06 2024-09-16T13:00:24.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966408+0000", 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.322e-06 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009095+0000", 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042687260999999997 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009276+0000", 2024-09-16T13:00:24.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000180345 2024-09-16T13:00:24.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009516+0000", 2024-09-16T13:00:24.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024073899999999999 2024-09-16T13:00:24.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.547 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48399 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501632588, 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043478389999999999, 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48399 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966412+0000", 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.03e-07 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966413+0000", 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2410000000000001e-06 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966416+0000", 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6330000000000002e-06 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966422+0000", 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2379999999999997e-06 2024-09-16T13:00:24.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009539+0000", 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043116865999999997 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009588+0000", 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9731999999999998e-05 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009890+0000", 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030137700000000003 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48400 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501618858000001, 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043788502, 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48400 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:24.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3999999999999997e-07 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966427+0000", 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.765e-06 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966434+0000", 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6989999999999997e-06 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966437+0000", 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1109999999999999e-06 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009969+0000", 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043532556 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010101+0000", 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0001317 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010213+0000", 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011233100000000001 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48401 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.556 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501604245999999, 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043965861000000002, 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48401 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966442+0000", 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8300000000000001e-06 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966455+0000", 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2927e-05 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966459+0000", 2024-09-16T13:00:24.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3850000000000004e-06 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010261+0000", 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043801305999999998 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010335+0000", 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4160999999999997e-05 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010405+0000", 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0716000000000005e-05 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48402 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.501579747999999, 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044179172000000003, 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.561 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48402 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969465+0000", 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003000689 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969469+0000", 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5739999999999997e-06 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969483+0000", 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3692e-05 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969488+0000", 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6450000000000004e-06 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010444+0000", 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040956030999999997 2024-09-16T13:00:24.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010542+0000", 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8486000000000005e-05 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010643+0000", 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101055 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.566 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.566 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:24.567 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:24.568 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op export --pgid 3.3 --file /var/log/ceph/exp.3.3.4 2024-09-16T13:00:24.568 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:24.562+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:24.667 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:24.662+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:24.725 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:24.725 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:24.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.725 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:91344 4.a 4:54eca525:::rbd_data.11b5403f243a.0000000000000044:head [stat,delete] snapc 1f=[1f] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:24.725 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.721801+0000", 2024-09-16T13:00:24.725 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0021709440000000002, 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002197222, 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 91344 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.721801+0000", 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.721801+0000", 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.721802+0000", 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:00:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.721807+0000", 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.408e-06 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.721828+0000", 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1152999999999999e-05 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.721843+0000", 2024-09-16T13:00:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4684999999999999e-05 2024-09-16T13:00:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.721885+0000", 2024-09-16T13:00:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2339999999999998e-05 2024-09-16T13:00:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.722003+0000", 2024-09-16T13:00:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011802 2024-09-16T13:00:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-16T13:00:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 1 2024-09-16T13:00:24.730 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:24.736 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:24.768 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:24.759+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:24.853 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:24.854 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:24.854 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:24.854 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:24.854 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:24.863 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:24.869 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:24.862+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:24.915 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:24.971 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:24.963+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:24.974 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:24.974 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:24.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:24.974 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:24.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.974 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T13:00:24.974 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:24.974 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.606508941999998, 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:00:24.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5580 4.12 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 56.277668212999998, 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034176870999999998, 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.980 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5580 2024-09-16T13:00:24.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.981 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:24.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:24.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693037+0000", 2024-09-16T13:00:24.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0650000000000001e-06 2024-09-16T13:00:24.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693045+0000", 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693070+0000", 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5341999999999999e-05 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693081+0000", 2024-09-16T13:00:24.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0611000000000001e-05 2024-09-16T13:00:24.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693182+0000", 2024-09-16T13:00:24.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100437 2024-09-16T13:00:24.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693309+0000", 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012741400000000001 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.703090+0000", 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097814800000000004 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:24.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726908+0000", 2024-09-16T13:00:24.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023817680000000001 2024-09-16T13:00:24.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727023+0000", 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011449400000000001 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727130+0000", 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010704099999999999 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727211+0000", 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1675000000000007e-05 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:18031 4.4 4:2649aabf:::rbd_data.11b51a3eb68b.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~5304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.810185708999999, 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94984657100000003, 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.988 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:24.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.989 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 18031 2024-09-16T13:00:24.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.989 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:24.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160519+0000", 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3029999999999998e-06 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160536+0000", 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7181999999999999e-05 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160564+0000", 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971e-05 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160576+0000", 2024-09-16T13:00:24.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160669+0000", 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3282999999999994e-05 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160680+0000", 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2024-09-16T13:00:24.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107188+0000", 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94650794999999999 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107644+0000", 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00045604499999999997 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.108257+0000", 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061324700000000003 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.109811+0000", 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015538749999999999 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110175+0000", 2024-09-16T13:00:24.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036419399999999998 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110285+0000", 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010938900000000001 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110364+0000", 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8689000000000005e-05 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.995 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55699 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.229744109, 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036277113999999999, 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:24.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55699 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:24.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740960+0000", 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.48e-06 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740967+0000", 2024-09-16T13:00:24.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8499999999999996e-06 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740984+0000", 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7555999999999999e-05 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740991+0000", 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6969999999999996e-06 2024-09-16T13:00:24.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741042+0000", 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0927000000000002e-05 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741116+0000", 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4016000000000004e-05 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741422+0000", 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030587600000000001 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.742556+0000", 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011342139999999999 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.776943+0000", 2024-09-16T13:00:25.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034386727999999998 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.777160+0000", 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021688499999999999 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.777236+0000", 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5884999999999996e-05 2024-09-16T13:00:25.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:63521 4.4 4:21a2b232:::rbd_data.11b5c2ee3542.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 16515072~7602176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.704090+0000", 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.2666128759999999, 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026833974999999999, 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:25.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.004 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 63521 2024-09-16T13:00:25.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.004 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.704090+0000", 2024-09-16T13:00:25.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.704090+0000", 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.704091+0000", 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6020000000000001e-06 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.708666+0000", 2024-09-16T13:00:25.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045744760000000001 2024-09-16T13:00:25.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.708696+0000", 2024-09-16T13:00:25.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0261999999999999e-05 2024-09-16T13:00:25.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.708707+0000", 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0516000000000001e-05 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.708789+0000", 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2851999999999999e-05 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.708951+0000", 2024-09-16T13:00:25.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000161056 2024-09-16T13:00:25.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:25.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.711570+0000", 2024-09-16T13:00:25.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026199169999999998 2024-09-16T13:00:25.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.719764+0000", 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081931090000000005 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.730805+0000", 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011041577 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:25.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.730867+0000", 2024-09-16T13:00:25.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1940000000000007e-05 2024-09-16T13:00:25.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.730924+0000", 2024-09-16T13:00:25.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6668e-05 2024-09-16T13:00:25.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.010 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.011 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:64609 4.e 4:7394d9c8:::rbd_object_map.11b5c2ee3542:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T13:00:25.011 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:25.011 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.7089765449999996, 2024-09-16T13:00:25.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.020058463, 2024-09-16T13:00:25.011 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.011 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:25.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.012 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 64609 2024-09-16T13:00:25.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.012 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:25.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261729+0000", 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4430000000000002e-06 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261741+0000", 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261780+0000", 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8722e-05 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261798+0000", 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8150000000000001e-05 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261927+0000", 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000129243 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261948+0000", 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1047e-05 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273712+0000", 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011763704999999999 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273720+0000", 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4309999999999994e-06 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280059+0000", 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.006338744 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280297+0000", 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000237667 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281645+0000", 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001348555 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281702+0000", 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6342999999999999e-05 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281785+0000", 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2782999999999997e-05 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.018 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86606 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 409600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505228+0000", 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46547453700000002, 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024062104000000001, 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86606 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.019 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505228+0000", 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505228+0000", 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505229+0000", 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9500000000000002e-07 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505231+0000", 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5170000000000002e-06 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505239+0000", 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0290000000000005e-06 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505251+0000", 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2153999999999999e-05 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529031+0000", 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023779432999999999 2024-09-16T13:00:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529116+0000", 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5047000000000001e-05 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529290+0000", 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000174329 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86607 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 417792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505259+0000", 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46544320700000003, 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024333345999999999, 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86607 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505259+0000", 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505259+0000", 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505260+0000", 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2399999999999998e-07 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505263+0000", 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6189999999999998e-06 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505271+0000", 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7150000000000004e-06 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505276+0000", 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9350000000000002e-06 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529319+0000", 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024042170000000002 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529425+0000", 2024-09-16T13:00:25.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106772 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529593+0000", 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016751099999999999 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86608 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.028 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505282+0000", 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.465420889, 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024564994999999999, 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86608 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505282+0000", 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505282+0000", 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505282+0000", 2024-09-16T13:00:25.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.68e-07 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505285+0000", 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5600000000000001e-06 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505293+0000", 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3790000000000006e-06 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505298+0000", 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1440000000000002e-06 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529646+0000", 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024347677000000002 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529721+0000", 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4523000000000005e-05 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529847+0000", 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012614399999999999 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86609 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 434176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.033 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505303+0000", 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.465399175, 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024723814, 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86609 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505303+0000", 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505303+0000", 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505304+0000", 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3900000000000003e-07 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505307+0000", 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.447e-06 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505315+0000", 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.568e-06 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505320+0000", 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0610000000000003e-06 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529873+0000", 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024552164000000001 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.529918+0000", 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5410000000000001e-05 2024-09-16T13:00:25.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530027+0000", 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109225 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86610 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 442368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505326+0000", 2024-09-16T13:00:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46537642899999998, 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024841141000000001, 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86610 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505326+0000", 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505326+0000", 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505327+0000", 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7999999999999995e-07 2024-09-16T13:00:25.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505329+0000", 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.413e-06 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505338+0000", 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5269999999999998e-06 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505343+0000", 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8729999999999998e-06 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530049+0000", 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024706451000000001 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530092+0000", 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2691000000000002e-05 2024-09-16T13:00:25.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530167+0000", 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5506000000000005e-05 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86611 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505348+0000", 2024-09-16T13:00:25.043 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46535482299999997, 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025097827, 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86611 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505348+0000", 2024-09-16T13:00:25.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505348+0000", 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505349+0000", 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7100000000000001e-07 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505351+0000", 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.52e-06 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505359+0000", 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3340000000000002e-06 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505364+0000", 2024-09-16T13:00:25.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7890000000000002e-06 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530191+0000", 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024826853999999999 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530259+0000", 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8429999999999994e-05 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530446+0000", 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000186229 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86612 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505370+0000", 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46533313700000001, 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025554922000000001, 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.048 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86612 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505370+0000", 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505370+0000", 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505370+0000", 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505373+0000", 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4779999999999998e-06 2024-09-16T13:00:25.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505381+0000", 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1200000000000002e-06 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505386+0000", 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9049999999999996e-06 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530573+0000", 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025187114 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530625+0000", 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1867999999999999e-05 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530924+0000", 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029988999999999998 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.052 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86613 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 466944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505391+0000", 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46531179700000003, 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02581516, 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86613 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505391+0000", 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505391+0000", 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505391+0000", 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7999999999999995e-07 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505394+0000", 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4880000000000001e-06 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505402+0000", 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3459999999999994e-06 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505414+0000", 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1712000000000001e-05 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.530960+0000", 2024-09-16T13:00:25.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025546105 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531043+0000", 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3207999999999993e-05 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531206+0000", 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162721 2024-09-16T13:00:25.057 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86614 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 475136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505420+0000", 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46528267299999998, 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026017459999999999, 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.058 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86614 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505420+0000", 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505420+0000", 2024-09-16T13:00:25.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505421+0000", 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000005e-07 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505423+0000", 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5450000000000002e-06 2024-09-16T13:00:25.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505432+0000", 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5150000000000006e-06 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505438+0000", 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0900000000000001e-06 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531228+0000", 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025790627 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531301+0000", 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2871000000000001e-05 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531437+0000", 2024-09-16T13:00:25.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013618699999999999 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86615 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 483328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505446+0000", 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46525682699999998, 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026284972, 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86615 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505446+0000", 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505446+0000", 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505446+0000", 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5199999999999996e-07 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505449+0000", 2024-09-16T13:00:25.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6469999999999998e-06 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505467+0000", 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7666e-05 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505479+0000", 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2223e-05 2024-09-16T13:00:25.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531477+0000", 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025997656000000001 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531591+0000", 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011464999999999999 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531731+0000", 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013947800000000001 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86616 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505486+0000", 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46521689999999999, 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026459280000000002, 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.068 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86616 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505486+0000", 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505486+0000", 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505486+0000", 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-07 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505494+0000", 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8890000000000002e-06 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505504+0000", 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2569999999999996e-06 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505509+0000", 2024-09-16T13:00:25.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1250000000000001e-06 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531755+0000", 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026246241 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531824+0000", 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8721999999999997e-05 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531945+0000", 2024-09-16T13:00:25.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012139200000000001 2024-09-16T13:00:25.073 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.073 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.073 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.073 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86617 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 499712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.073 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505514+0000", 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46518817800000001, 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026600615000000001, 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86617 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505514+0000", 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505514+0000", 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505517+0000", 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2929999999999999e-06 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505519+0000", 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6819999999999999e-06 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505527+0000", 2024-09-16T13:00:25.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9370000000000004e-06 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505533+0000", 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1359999999999996e-06 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.531976+0000", 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026443497999999999 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.532021+0000", 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5333000000000001e-05 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.532115+0000", 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3735999999999998e-05 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86618 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505538+0000", 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.46516445000000001, 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026790412, 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86618 2024-09-16T13:00:25.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505538+0000", 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505538+0000", 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505539+0000", 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8100000000000002e-07 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505541+0000", 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3779999999999999e-06 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505550+0000", 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3750000000000003e-06 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505555+0000", 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0030000000000002e-06 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.532143+0000", 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026587989999999999 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.532235+0000", 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2474999999999998e-05 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.532329+0000", 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3410000000000002e-05 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:86619 4.5 4:a1402939:::rbd_data.11b515823229.0000000000000030:1e [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:24.505560+0000", 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.465142526, 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026924213999999998, 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:25.084 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 86619 2024-09-16T13:00:25.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.085 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:25.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:25.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505560+0000", 2024-09-16T13:00:25.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:25.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505560+0000", 2024-09-16T13:00:25.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:25.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:25.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505561+0000", 2024-09-16T13:00:25.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5799999999999999e-07 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505563+0000", 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4439999999999998e-06 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505571+0000", 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2889999999999998e-06 2024-09-16T13:00:25.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.505576+0000", 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7330000000000003e-06 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.532356+0000", 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026780014000000001 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.532397+0000", 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0541999999999999e-05 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:24.532484+0000", 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7633999999999995e-05 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.089 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:25.090 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:25.090 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:25.091 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:25.092 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:25.084+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:25.146 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:25.176 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:25.191 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:25.186+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:25.220 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:25.227 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:25.228 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:25.228 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:25.228 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:25.237 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:25.250 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:25.292 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:25.284+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:25.309 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T13:00:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:25.321 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:25.321 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:25.321 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:25.321 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:25.321 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:25.321 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:25.321 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:25.321 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:25.323 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:25.324 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:25.334 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:25.334 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:25.343 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:25.393 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:25.384+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:25.462 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:25.462 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:25.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:25.494 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:25.488+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:25.595 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:25.588+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:25.600 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:25.600 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:25.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:25.700 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:25.688+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:25.769 INFO:teuthology.orchestra.run.smithi136.stderr:Exporting 3.3 info 3.3( empty local-lis/les=23/24 n=0 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-16T13:00:25.775 INFO:teuthology.orchestra.run.smithi136.stderr:Export successful 2024-09-16T13:00:25.799 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:25.792+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:25.800 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:25.800 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:25.814 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:25.898 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:25.890+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:25.998 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:25.992+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:26.011 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:26.012 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:26.022 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:26.099 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:26.094+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:26.155 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:26.155 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:26.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:26.200 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:26.192+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:26.301 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:26.294+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:26.305 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:26.305 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:26.315 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T13:00:26.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:26.347 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --force --op remove --pgid 3.3 2024-09-16T13:00:26.402 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:26.396+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:26.503 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:26.496+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:26.510 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:26.510 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:26.521 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:26.603 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:26.598+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:26.703 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:26.698+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:26.709 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:26.709 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:26.721 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:26.804 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:26.798+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:26.871 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:26.871 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:26.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:26.905 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:26.898+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:27.006 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:26.998+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:27.030 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:27.030 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:27.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:27.106 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:27.100+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:27.196 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:27.196 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:27.209 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:27.200+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:27.211 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:27.308 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:27.300+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:27.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:27.339 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:27.339 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:27.350 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:27.409 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:27.400+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:27.509 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:27.504+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:27.520 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: marking collection for removal 2024-09-16T13:00:27.541 INFO:teuthology.orchestra.run.smithi136.stdout:setting '_remove' omap key 2024-09-16T13:00:27.541 INFO:teuthology.orchestra.run.smithi136.stdout:finish_remove_pgs 3.3_head removing 3.3 2024-09-16T13:00:27.549 INFO:teuthology.orchestra.run.smithi136.stdout:Remove successful 2024-09-16T13:00:27.553 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:27.553 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:27.555 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:27.565 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:27.590 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:27.612 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:27.604+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:27.627 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:27.672 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:27.683 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:27.683 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:27.683 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:27.683 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:27.694 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:27.701 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:27.711 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:27.706+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:27.728 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:27.728 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:27.729 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:27.729 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:27.729 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:27.729 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:27.729 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:27.729 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:27.729 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:27.732 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:27.732 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:27.794 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:27.794 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:27.794 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:27.795 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:27.795 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:27.804 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:27.812 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:27.804+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:27.913 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:27.904+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8419 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.516701965000003, 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040293592000000003, 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.923 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8419 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402894+0000", 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402907+0000", 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402940+0000", 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3436000000000003e-05 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402953+0000", 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2840000000000001e-05 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403104+0000", 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150712 2024-09-16T13:00:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403202+0000", 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8403e-05 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.411015+0000", 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078133079999999997 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443018+0000", 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032002348999999999 2024-09-16T13:00:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443075+0000", 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7703e-05 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443141+0000", 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5705999999999997e-05 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443185+0000", 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4152e-05 2024-09-16T13:00:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953469606999999, 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039945895000000002, 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48384 2024-09-16T13:00:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966126+0000", 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1110000000000002e-06 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966134+0000", 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9740000000000002e-06 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966139+0000", 2024-09-16T13:00:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4270000000000001e-06 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.005829+0000", 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039690007999999999 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.005928+0000", 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9184000000000004e-05 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006070+0000", 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014171100000000001 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953449976000002, 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040120731, 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48385 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.63e-07 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966146+0000", 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.018e-06 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966154+0000", 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9999999999999996e-06 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966169+0000", 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5395000000000001e-05 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006118+0000", 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039948637000000002 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006181+0000", 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3001000000000002e-05 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006264+0000", 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3217e-05 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953418852999999, 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040378561, 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48386 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5199999999999996e-07 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966178+0000", 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.373e-06 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966186+0000", 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3259999999999996e-06 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966202+0000", 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6333000000000001e-05 2024-09-16T13:00:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006311+0000", 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040108447999999998 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006411+0000", 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100141 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006553+0000", 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014218799999999999 2024-09-16T13:00:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953385321999999, 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040537542000000003, 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48387 2024-09-16T13:00:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966209+0000", 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4300000000000003e-07 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966211+0000", 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.418e-06 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966219+0000", 2024-09-16T13:00:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.627e-06 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966223+0000", 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3019999999999996e-06 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006601+0000", 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040377953000000001 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006665+0000", 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3592999999999999e-05 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006746+0000", 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1106000000000006e-05 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953365724000001, 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040793276000000003, 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48388 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5700000000000002e-07 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966230+0000", 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0499999999999999e-06 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966246+0000", 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5543e-05 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966250+0000", 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0840000000000002e-06 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006794+0000", 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040543691999999999 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006901+0000", 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107754 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007021+0000", 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119596 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953338324000001, 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040958857000000001, 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48389 2024-09-16T13:00:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966256+0000", 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8100000000000003e-07 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966258+0000", 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3829999999999999e-06 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966268+0000", 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8379999999999997e-06 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966272+0000", 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9940000000000002e-06 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007069+0000", 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040796897999999998 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007132+0000", 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3084999999999997e-05 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007214+0000", 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2177999999999998e-05 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953315845999999, 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041219483000000001, 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48390 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4700000000000002e-07 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966280+0000", 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3e-06 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966286+0000", 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.57e-06 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966288+0000", 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4439999999999998e-06 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007262+0000", 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040973242 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007351+0000", 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9339999999999995e-05 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007497+0000", 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014614000000000001 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953294183000001, 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041394590000000002, 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48391 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966300+0000", 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966301+0000", 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5510000000000001e-06 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966305+0000", 2024-09-16T13:00:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1849999999999997e-06 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966310+0000", 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.211e-06 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007546+0000", 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041236341000000003 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007610+0000", 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3534999999999994e-05 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007694+0000", 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4407999999999996e-05 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48392 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 794624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953281529000002, 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041645095, 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48392 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3599999999999999e-07 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966314+0000", 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7689999999999999e-06 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966322+0000", 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9340000000000006e-06 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966325+0000", 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5579999999999999e-06 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007742+0000", 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041417127999999998 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007840+0000", 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8199e-05 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007957+0000", 2024-09-16T13:00:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011717099999999999 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48393 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953266332999998, 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041849738999999997, 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48393 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966328+0000", 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966340+0000", 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2604e-05 2024-09-16T13:00:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966344+0000", 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1989999999999999e-06 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966348+0000", 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4039999999999999e-06 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008030+0000", 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041682036999999998 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008093+0000", 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3511999999999998e-05 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008177+0000", 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3561999999999997e-05 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48394 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953242874000001, 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042095443000000003, 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48394 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966352+0000", 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.406e-06 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966356+0000", 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966362+0000", 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2299999999999996e-06 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008232+0000", 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041869361000000001 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008320+0000", 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8492000000000006e-05 2024-09-16T13:00:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008446+0000", 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012591800000000001 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48395 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953227333000001, 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042265693999999999, 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48395 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966367+0000", 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966368+0000", 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9420000000000002e-06 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966372+0000", 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9049999999999999e-06 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966375+0000", 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4329999999999998e-06 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008483+0000", 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042108244000000003 2024-09-16T13:00:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008546+0000", 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2973999999999997e-05 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008632+0000", 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.585e-05 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48396 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953216246, 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042505605000000002, 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48396 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966378+0000", 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966379+0000", 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.147e-06 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966382+0000", 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966385+0000", 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.43e-06 2024-09-16T13:00:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008677+0000", 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042292191 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008765+0000", 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8027000000000003e-05 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008883+0000", 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117817 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48397 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953206340000001, 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042627777999999998, 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48397 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966388+0000", 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966389+0000", 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.144e-06 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966392+0000", 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5939999999999998e-06 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966398+0000", 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3959999999999997e-06 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008904+0000", 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042505730999999998 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008953+0000", 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9786000000000001e-05 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009015+0000", 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1580000000000003e-05 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48398 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953192997999999, 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043115957000000003, 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48398 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2500000000000001e-07 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966402+0000", 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3120000000000001e-06 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966406+0000", 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.653e-06 2024-09-16T13:00:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966408+0000", 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.322e-06 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009095+0000", 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042687260999999997 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009276+0000", 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000180345 2024-09-16T13:00:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009516+0000", 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024073899999999999 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48399 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953182264999999, 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043478389999999999, 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48399 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966412+0000", 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.03e-07 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966413+0000", 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2410000000000001e-06 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966416+0000", 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6330000000000002e-06 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966422+0000", 2024-09-16T13:00:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2379999999999997e-06 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009539+0000", 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043116865999999997 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009588+0000", 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9731999999999998e-05 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009890+0000", 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030137700000000003 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48400 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953168535, 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043788502, 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48400 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3999999999999997e-07 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966427+0000", 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.765e-06 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966434+0000", 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6989999999999997e-06 2024-09-16T13:00:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966437+0000", 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1109999999999999e-06 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009969+0000", 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043532556 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010101+0000", 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0001317 2024-09-16T13:00:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010213+0000", 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011233100000000001 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48401 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953153922999999, 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043965861000000002, 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48401 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966442+0000", 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8300000000000001e-06 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966455+0000", 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2927e-05 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966459+0000", 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3850000000000004e-06 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010261+0000", 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043801305999999998 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010335+0000", 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4160999999999997e-05 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010405+0000", 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0716000000000005e-05 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48402 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.953129425, 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044179172000000003, 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48402 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969465+0000", 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003000689 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969469+0000", 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5739999999999997e-06 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969483+0000", 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3692e-05 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969488+0000", 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6450000000000004e-06 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010444+0000", 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040956030999999997 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010542+0000", 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8486000000000005e-05 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010643+0000", 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101055 2024-09-16T13:00:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.013 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:28.014 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:28.015 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:28.008+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:28.089 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.3.4 2024-09-16T13:00:28.115 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:28.108+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:28.120 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:153719 4.5 4:a322636d:::rbd_data.11b515823229.000000000000017e:1e [read 475136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:28.115807+0000", 2024-09-16T13:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0028139739999999999, 2024-09-16T13:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028508069999999999, 2024-09-16T13:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153719 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115807+0000", 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115807+0000", 2024-09-16T13:00:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115808+0000", 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1799999999999995e-07 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115810+0000", 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4880000000000001e-06 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115816+0000", 2024-09-16T13:00:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.964e-06 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115820+0000", 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7060000000000002e-06 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.118365+0000", 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00254514 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.118560+0000", 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019448800000000001 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:153720 4.5 4.b6c644c5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:28.115824+0000", 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0027974229999999998, 2024-09-16T13:00:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029292540000000001, 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153720 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115824+0000", 2024-09-16T13:00:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115824+0000", 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115824+0000", 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2599999999999998e-07 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115827+0000", 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1900000000000002e-06 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115832+0000", 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1159999999999998e-06 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115835+0000", 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4699999999999998e-06 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:153721 4.5 4.b6c644c5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:28.115839+0000", 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.002782587, 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029836900000000002, 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153721 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115839+0000", 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115839+0000", 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115839+0000", 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3799999999999998e-07 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115841+0000", 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9e-06 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115846+0000", 2024-09-16T13:00:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9960000000000001e-06 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115849+0000", 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3699999999999999e-06 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:153722 4.5 4.b6c644c5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:28.115853+0000", 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0027681849999999998, 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030859080000000001, 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153722 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115853+0000", 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115853+0000", 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115854+0000", 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6600000000000002e-07 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115856+0000", 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9300000000000002e-06 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115861+0000", 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2059999999999998e-06 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115864+0000", 2024-09-16T13:00:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3170000000000002e-06 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:153723 4.5 4.b6c644c5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:28.115867+0000", 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0027540099999999999, 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003211527, 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153723 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115867+0000", 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115867+0000", 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115868+0000", 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6400000000000003e-07 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115870+0000", 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9999999999999999e-06 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115875+0000", 2024-09-16T13:00:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2519999999999999e-06 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115879+0000", 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9299999999999996e-06 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:153724 4.5 4.b6c644c5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:28.115897+0000", 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.002724349, 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033163889999999999, 2024-09-16T13:00:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153724 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115897+0000", 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115897+0000", 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115897+0000", 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8099999999999998e-07 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115899+0000", 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6649999999999999e-06 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115903+0000", 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1860000000000002e-06 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:28.115906+0000", 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9869999999999999e-06 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-16T13:00:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 6 2024-09-16T13:00:28.143 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:28.143 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:28.215 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:28.210+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:28.247 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:28.247 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:28.247 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:28.247 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:28.247 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:28.256 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:28.316 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:28.310+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:28.330 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T13:00:28.402 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:28.402 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:28.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:28.402 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:28.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.402 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T13:00:28.402 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:28.402 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 61.03332417, 2024-09-16T13:00:28.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T13:00:28.402 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T13:00:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T13:00:28.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:5580 4.12 4:4adc24a4:::rbd_data.11b53fd98b75.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.704483441000001, 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034176870999999998, 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.407 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5580 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693034+0000", 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693037+0000", 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0650000000000001e-06 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693045+0000", 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6319999999999997e-06 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693070+0000", 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5341999999999999e-05 2024-09-16T13:00:28.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693081+0000", 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0611000000000001e-05 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693182+0000", 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100437 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.693309+0000", 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012741400000000001 2024-09-16T13:00:28.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.703090+0000", 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097814800000000004 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.726908+0000", 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023817680000000001 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727023+0000", 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011449400000000001 2024-09-16T13:00:28.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727130+0000", 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010704099999999999 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:28.727211+0000", 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1675000000000007e-05 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.412 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:18031 4.4 4:2649aabf:::rbd_data.11b51a3eb68b.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~5304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.237000936999998, 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94984657100000003, 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 18031 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160519+0000", 2024-09-16T13:00:28.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3029999999999998e-06 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160536+0000", 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7181999999999999e-05 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160564+0000", 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971e-05 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160576+0000", 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160669+0000", 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3282999999999994e-05 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160680+0000", 2024-09-16T13:00:28.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107188+0000", 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94650794999999999 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107644+0000", 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00045604499999999997 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.108257+0000", 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061324700000000003 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.109811+0000", 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015538749999999999 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110175+0000", 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036419399999999998 2024-09-16T13:00:28.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110285+0000", 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010938900000000001 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110364+0000", 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8689000000000005e-05 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:55699 4.a 4:55c48c7b:::rbd_object_map.11b54010928d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.656559337000001, 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036277113999999999, 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 55699 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740959+0000", 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740960+0000", 2024-09-16T13:00:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.48e-06 2024-09-16T13:00:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740967+0000", 2024-09-16T13:00:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8499999999999996e-06 2024-09-16T13:00:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740984+0000", 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7555999999999999e-05 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.740991+0000", 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6969999999999996e-06 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741042+0000", 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0927000000000002e-05 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741116+0000", 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4016000000000004e-05 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.741422+0000", 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030587600000000001 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.742556+0000", 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011342139999999999 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.776943+0000", 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034386727999999998 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.777160+0000", 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021688499999999999 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:14.777236+0000", 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5884999999999996e-05 2024-09-16T13:00:28.425 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:64609 4.e 4:7394d9c8:::rbd_object_map.11b5c2ee3542:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1357917729999993, 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.020058463, 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.426 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 64609 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:28.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261729+0000", 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4430000000000002e-06 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261741+0000", 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261780+0000", 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8722e-05 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261798+0000", 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8150000000000001e-05 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261927+0000", 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000129243 2024-09-16T13:00:28.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261948+0000", 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1047e-05 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273712+0000", 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011763704999999999 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:28.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273720+0000", 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4309999999999994e-06 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280059+0000", 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.006338744 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280297+0000", 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000237667 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281645+0000", 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001348555 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281702+0000", 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6342999999999999e-05 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281785+0000", 2024-09-16T13:00:28.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2782999999999997e-05 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150586 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.894920+0000", 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50259834199999998, 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033217517000000002, 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.433 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150586 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894920+0000", 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894920+0000", 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894920+0000", 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8599999999999999e-07 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894922+0000", 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7659999999999999e-06 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894927+0000", 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3879999999999999e-06 2024-09-16T13:00:28.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894932+0000", 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1540000000000001e-06 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.928002+0000", 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033069381000000002 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.928074+0000", 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2050000000000003e-05 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.928137+0000", 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3392000000000006e-05 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150587 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 227328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.894936+0000", 2024-09-16T13:00:28.437 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50258188000000004, 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034651214999999999, 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150587 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894936+0000", 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894936+0000", 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894937+0000", 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1099999999999995e-07 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894939+0000", 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9190000000000002e-06 2024-09-16T13:00:28.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894944+0000", 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5539999999999997e-06 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894948+0000", 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4139999999999998e-06 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.928559+0000", 2024-09-16T13:00:28.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033611121000000001 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.928740+0000", 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000180963 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929587+0000", 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084763299999999998 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150588 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.894953+0000", 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50256535700000005, 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033988443, 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150588 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894953+0000", 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.894953+0000", 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895737+0000", 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078419799999999995 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895742+0000", 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8509999999999998e-06 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895753+0000", 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1613999999999999e-05 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895759+0000", 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.981e-06 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.928837+0000", 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033078321000000001 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.928887+0000", 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9197999999999999e-05 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.928941+0000", 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4280000000000002e-05 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150589 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.895765+0000", 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50175251799999998, 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033283741999999998, 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150589 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.446 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895765+0000", 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895765+0000", 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895766+0000", 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8899999999999998e-07 2024-09-16T13:00:28.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895768+0000", 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1679999999999998e-06 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895775+0000", 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5549999999999999e-06 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895779+0000", 2024-09-16T13:00:28.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7340000000000002e-06 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.928960+0000", 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.03318173 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929006+0000", 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5495999999999999e-05 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929049+0000", 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3269999999999997e-05 2024-09-16T13:00:28.449 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150590 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 251904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.895794+0000", 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50172356399999996, 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033364086000000001, 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150590 2024-09-16T13:00:28.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895794+0000", 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895794+0000", 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895795+0000", 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8400000000000004e-07 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895797+0000", 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.024e-06 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895804+0000", 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9739999999999996e-06 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895808+0000", 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9330000000000003e-06 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929071+0000", 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033263248000000002 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929119+0000", 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7491999999999999e-05 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929158+0000", 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9731000000000003e-05 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150591 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 260096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.895812+0000", 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50170551299999999, 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033437685000000002, 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150591 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895812+0000", 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895812+0000", 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895813+0000", 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8799999999999998e-07 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895815+0000", 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.131e-06 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895821+0000", 2024-09-16T13:00:28.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0950000000000001e-06 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895825+0000", 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7440000000000001e-06 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929172+0000", 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033347053000000001 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929215+0000", 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2752e-05 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929250+0000", 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5522000000000003e-05 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150592 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 268288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.895829+0000", 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.501689041, 2024-09-16T13:00:28.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033550498999999998, 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150592 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895829+0000", 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895829+0000", 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895829+0000", 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3500000000000002e-07 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895831+0000", 2024-09-16T13:00:28.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8959999999999999e-06 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895837+0000", 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3530000000000002e-06 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895847+0000", 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0271999999999999e-05 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929273+0000", 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033426495000000001 2024-09-16T13:00:28.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929341+0000", 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7228000000000001e-05 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929379+0000", 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8819999999999998e-05 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150593 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 276480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.895851+0000", 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50166694300000003, 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033672995999999997, 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150593 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895851+0000", 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895851+0000", 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895851+0000", 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895854+0000", 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4990000000000001e-06 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895860+0000", 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.054e-06 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895863+0000", 2024-09-16T13:00:28.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3280000000000002e-06 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929393+0000", 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033530031000000002 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929453+0000", 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9525999999999999e-05 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929524+0000", 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1136999999999993e-05 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150594 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 284672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.895868+0000", 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50164979499999995, 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033851843, 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.467 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150594 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895868+0000", 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895868+0000", 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895869+0000", 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5299999999999999e-07 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895871+0000", 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1100000000000001e-06 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895877+0000", 2024-09-16T13:00:28.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4980000000000003e-06 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895881+0000", 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0520000000000003e-06 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929547+0000", 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033666199000000001 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929652+0000", 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010508 2024-09-16T13:00:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929720+0000", 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7451000000000002e-05 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150595 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 292864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.895887+0000", 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50163125500000005, 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034051076, 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.471 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150595 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895887+0000", 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895887+0000", 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895887+0000", 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7500000000000001e-07 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895889+0000", 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8050000000000001e-06 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895894+0000", 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.916e-06 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895897+0000", 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2160000000000002e-06 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929783+0000", 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033885765999999998 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929857+0000", 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4642999999999997e-05 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929938+0000", 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0354999999999999e-05 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150596 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 301056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.895925+0000", 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50159301499999998, 2024-09-16T13:00:28.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034227457000000003, 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150596 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895925+0000", 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895925+0000", 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895926+0000", 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7400000000000002e-07 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895928+0000", 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.384e-06 2024-09-16T13:00:28.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895934+0000", 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1800000000000001e-06 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895938+0000", 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0470000000000004e-06 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.929971+0000", 2024-09-16T13:00:28.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034032771000000003 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930057+0000", 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5593000000000005e-05 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930152+0000", 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5707999999999999e-05 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150597 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 309248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.895942+0000", 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50157546200000003, 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034387615000000003, 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150597 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.480 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895942+0000", 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895942+0000", 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895943+0000", 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1799999999999995e-07 2024-09-16T13:00:28.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895950+0000", 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5950000000000004e-06 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895956+0000", 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0079999999999999e-06 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895959+0000", 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4029999999999999e-06 2024-09-16T13:00:28.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930183+0000", 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034223933999999998 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930262+0000", 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8770999999999995e-05 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930330+0000", 2024-09-16T13:00:28.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8386000000000006e-05 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150598 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 317440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.895963+0000", 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.501555321, 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034566351000000002, 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150598 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895963+0000", 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.895963+0000", 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896013+0000", 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9938000000000002e-05 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896015+0000", 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8040000000000002e-06 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896023+0000", 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2169999999999997e-06 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896027+0000", 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5549999999999996e-06 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930359+0000", 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034332019999999998 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930441+0000", 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1409999999999995e-05 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930529+0000", 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8406999999999996e-05 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150599 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 325632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.488 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.896032+0000", 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50148588400000005, 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034746132999999998, 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150599 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896032+0000", 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896032+0000", 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896111+0000", 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9252999999999995e-05 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896115+0000", 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3510000000000002e-06 2024-09-16T13:00:28.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896122+0000", 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.418e-06 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896126+0000", 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.177e-06 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930621+0000", 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034494552999999997 2024-09-16T13:00:28.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930702+0000", 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1290000000000003e-05 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930778+0000", 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6091e-05 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.492 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:150600 4.1f 4:fce195fa:::rbd_data.11b515823229.0000000000000159:1e [read 333824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:27.896131+0000", 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50138702099999999, 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034864801000000001, 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150600 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896131+0000", 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.896131+0000", 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.897477+0000", 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001346051 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.897482+0000", 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2669999999999997e-06 2024-09-16T13:00:28.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.897492+0000", 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3409999999999992e-06 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.897498+0000", 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9800000000000003e-06 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930835+0000", 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033337594999999998 2024-09-16T13:00:28.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930916+0000", 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1169999999999997e-05 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:27.930996+0000", 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9396999999999999e-05 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.496 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:28.497 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-16T13:00:28.498 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:28.499 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:28.492+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:28.598 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:28.594+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:28.612 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:28.612 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:28.612 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:28.612 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:28.626 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:28.699 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:28.692+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:28.725 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:28.725 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:28.726 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:28.726 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:28.726 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:28.734 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:28.800 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:28.794+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:28.844 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 103.289453435, 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.46441595899999, 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010976025, 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 45 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374759+0000", 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374760+0000", 2024-09-16T13:00:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.13e-06 2024-09-16T13:00:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374769+0000", 2024-09-16T13:00:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5699999999999999e-06 2024-09-16T13:00:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374819+0000", 2024-09-16T13:00:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9882000000000001e-05 2024-09-16T13:00:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374828+0000", 2024-09-16T13:00:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-16T13:00:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382329+0000", 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075009009999999999 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382403+0000", 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4208999999999994e-05 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.382514+0000", 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111276 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385533+0000", 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030190080000000001 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385564+0000", 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0926000000000002e-05 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385666+0000", 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101791 2024-09-16T13:00:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385735+0000", 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8541999999999995e-05 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.46433638800001, 2024-09-16T13:00:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:00:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.464289132, 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:00:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.464086196, 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:00:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.464054934, 2024-09-16T13:00:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:00:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:00:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.46401917199999, 2024-09-16T13:00:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:00:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.463824542, 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:00:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:00:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.463777697, 2024-09-16T13:00:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:00:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:00:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.919501526, 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:00:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.919447619, 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:00:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:00:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.919405314, 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:00:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.91924688899999, 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:00:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:00:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.919197408, 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:00:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:00:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.91906683800001, 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:00:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:00:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.91888532500001, 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:00:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:00:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.108357658, 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:00:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.10820870400001, 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:00:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:00:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64507 4.1e e83/31 4:7f7c7834:::rbd_data.11b5c2ee3542.0000000000000008:head v 83'225, mlcod=81'223)", 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.068795417, 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021284401000000001, 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770380+0000", 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8999999999999998e-07 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777141+0000", 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067617670000000001 2024-09-16T13:00:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777157+0000", 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777167+0000", 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777267+0000", 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100364 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777304+0000", 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7165000000000001e-05 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791601+0000", 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014297275999999999 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791664+0000", 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2217000000000004e-05 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64509 4.15 e83/31 4:ad2e18f4:::rbd_data.11b5c2ee3542.0000000000000009:head v 83'183, mlcod=81'180)", 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.062037515, 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014054395000000001, 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777138+0000", 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778534+0000", 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013956769999999999 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778555+0000", 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1046000000000001e-05 2024-09-16T13:00:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778564+0000", 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9290000000000004e-06 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778633+0000", 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9363000000000007e-05 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778666+0000", 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135000000000003e-05 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791145+0000", 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012479743 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791192+0000", 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6156000000000003e-05 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:28.963 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:28.964 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:28.967 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:28.956+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:29.066 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:29.060+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:29.122 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:29.122 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:29.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:29.166 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:29.158+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:29.266 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:29.260+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:29.286 INFO:teuthology.orchestra.run.smithi136.stdout:get_pg_num_history pg_num_history pg_num_history(e83 pg_nums {1={15=1},2={18=8},3={23=4},4={31=32}} deleted_pools ) 2024-09-16T13:00:29.286 INFO:teuthology.orchestra.run.smithi136.stdout:Importing pgid 3.3 2024-09-16T13:00:29.286 INFO:teuthology.orchestra.run.smithi136.stdout:write_pg epoch 65 info 3.3( empty local-lis/les=23/24 n=0 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-16T13:00:29.287 INFO:teuthology.orchestra.run.smithi136.stdout:Import successful 2024-09-16T13:00:29.297 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:29.297 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:29.306 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T13:00:29.367 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:29.360+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:29.417 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:29.417 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719396158, 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:00:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:00:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:00:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:00:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:00:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719381132000001, 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:29.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:00:29.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.71936511, 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:00:29.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:00:29.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719349014000001, 2024-09-16T13:00:29.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:00:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:00:29.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:00:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:00:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:00:29.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719333124, 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.438 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.442 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719317454, 2024-09-16T13:00:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:00:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:00:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:00:29.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719301707, 2024-09-16T13:00:29.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:00:29.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:00:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719285919000001, 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:00:29.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:00:29.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:00:29.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:00:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719270597, 2024-09-16T13:00:29.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:00:29.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:00:29.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719250671999999, 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.462 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:00:29.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:00:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:00:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:00:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:00:29.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.466 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719234724, 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.467 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:00:29.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.471 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719219562999999, 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:00:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:00:29.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719203915, 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.481 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719188817999999, 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:00:29.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:00:29.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719173555999999, 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.71915781, 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:29.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719142213, 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:00:29.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:00:29.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.501 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.502 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.502 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:29.502 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719126909, 2024-09-16T13:00:29.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:00:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719111471, 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:00:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:00:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.719095758, 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:00:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:29.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:00:29.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:00:29.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.517 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.517 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.517 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:29.519 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:29.519 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:29.512+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:29.540 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:29.550 INFO:teuthology.orchestra.run.smithi042.stdout:4651 trunc from 0xcce5d04 to 0x3a63588 2024-09-16T13:00:29.619 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:29.614+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:29.695 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177013 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654118+0000", 2024-09-16T13:00:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034428907000000002, 2024-09-16T13:00:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035292726000000003, 2024-09-16T13:00:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177013 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654118+0000", 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654118+0000", 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654118+0000", 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8200000000000001e-07 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654120+0000", 2024-09-16T13:00:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0729999999999999e-06 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654131+0000", 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0876000000000001e-05 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654135+0000", 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8800000000000001e-06 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177014 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654139+0000", 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034407364000000003, 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035350316999999999, 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177014 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654139+0000", 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654139+0000", 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654140+0000", 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654142+0000", 2024-09-16T13:00:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0770000000000002e-06 2024-09-16T13:00:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654148+0000", 2024-09-16T13:00:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1929999999999998e-06 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654152+0000", 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.737e-06 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177015 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654156+0000", 2024-09-16T13:00:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034390916000000001, 2024-09-16T13:00:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035391461999999999, 2024-09-16T13:00:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177015 2024-09-16T13:00:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654156+0000", 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654156+0000", 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654156+0000", 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8599999999999999e-07 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654164+0000", 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1130000000000001e-06 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654173+0000", 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8210000000000008e-06 2024-09-16T13:00:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654177+0000", 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9199999999999997e-06 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177016 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654181+0000", 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034365389000000003, 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035420370999999999, 2024-09-16T13:00:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177016 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654181+0000", 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654181+0000", 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654182+0000", 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.27e-07 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654184+0000", 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.266e-06 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654190+0000", 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0220000000000001e-06 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654194+0000", 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5820000000000002e-06 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177017 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654198+0000", 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034349085000000001, 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035456311999999997, 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177017 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654198+0000", 2024-09-16T13:00:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654198+0000", 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654198+0000", 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654200+0000", 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7570000000000001e-06 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654209+0000", 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8820000000000007e-06 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654212+0000", 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5410000000000001e-06 2024-09-16T13:00:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177018 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654216+0000", 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034330370999999998, 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035491345000000001, 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177018 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654216+0000", 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654216+0000", 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654217+0000", 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6300000000000005e-07 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654219+0000", 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654228+0000", 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1400000000000006e-06 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654232+0000", 2024-09-16T13:00:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.856e-06 2024-09-16T13:00:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177019 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654236+0000", 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034310632000000001, 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035534441999999999, 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177019 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654236+0000", 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654236+0000", 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654236+0000", 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654239+0000", 2024-09-16T13:00:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4719999999999998e-06 2024-09-16T13:00:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654245+0000", 2024-09-16T13:00:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4649999999999999e-06 2024-09-16T13:00:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654250+0000", 2024-09-16T13:00:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0960000000000003e-06 2024-09-16T13:00:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:176998 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 5666816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.653801+0000", 2024-09-16T13:00:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034745606999999998, 2024-09-16T13:00:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036031242999999998, 2024-09-16T13:00:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 176998 2024-09-16T13:00:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653801+0000", 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653801+0000", 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653802+0000", 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.32e-07 2024-09-16T13:00:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653804+0000", 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653812+0000", 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8389999999999996e-06 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653816+0000", 2024-09-16T13:00:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.929e-06 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.689311+0000", 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.035494824000000001 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.689377+0000", 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6179000000000005e-05 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:176999 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.653820+0000", 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034726281999999997, 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036075019, 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 176999 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653820+0000", 2024-09-16T13:00:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653820+0000", 2024-09-16T13:00:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653821+0000", 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653823+0000", 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1050000000000002e-06 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653832+0000", 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7630000000000006e-06 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653835+0000", 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7359999999999999e-06 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177000 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.653840+0000", 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034706800000000003, 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036122622, 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177000 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653840+0000", 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653840+0000", 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653840+0000", 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.46e-07 2024-09-16T13:00:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653843+0000", 2024-09-16T13:00:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3089999999999998e-06 2024-09-16T13:00:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653851+0000", 2024-09-16T13:00:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4009999999999997e-06 2024-09-16T13:00:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653855+0000", 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8859999999999997e-06 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177001 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.653860+0000", 2024-09-16T13:00:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034687098999999999, 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036156793, 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177001 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653860+0000", 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653860+0000", 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653860+0000", 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8999999999999997e-07 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653863+0000", 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653872+0000", 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1870000000000003e-06 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653875+0000", 2024-09-16T13:00:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177002 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.653880+0000", 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034667079000000003, 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036189404000000001, 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177002 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653880+0000", 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653880+0000", 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653880+0000", 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653882+0000", 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.08e-06 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653890+0000", 2024-09-16T13:00:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0949999999999996e-06 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653894+0000", 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8539999999999999e-06 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177003 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.653904+0000", 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034642602000000002, 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036219137999999998, 2024-09-16T13:00:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177003 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653904+0000", 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653904+0000", 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653905+0000", 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653907+0000", 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3760000000000002e-06 2024-09-16T13:00:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653916+0000", 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2650000000000002e-06 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653920+0000", 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177004 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.653924+0000", 2024-09-16T13:00:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034622361999999997, 2024-09-16T13:00:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036252529999999998, 2024-09-16T13:00:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177004 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653924+0000", 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653924+0000", 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653925+0000", 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4499999999999997e-07 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653927+0000", 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.074e-06 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653935+0000", 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5960000000000004e-06 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653939+0000", 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5260000000000002e-06 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177005 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.653944+0000", 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034603028000000001, 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036297601999999998, 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177005 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653944+0000", 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653944+0000", 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653944+0000", 2024-09-16T13:00:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4099999999999999e-07 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653946+0000", 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.12e-06 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653965+0000", 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8862999999999999e-05 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653970+0000", 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6530000000000002e-06 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177006 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.653975+0000", 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034571936999999997, 2024-09-16T13:00:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036320914000000003, 2024-09-16T13:00:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177006 2024-09-16T13:00:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653975+0000", 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653975+0000", 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653975+0000", 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653978+0000", 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.649e-06 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653989+0000", 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1452e-05 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653994+0000", 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2350000000000001e-06 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177007 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.653999+0000", 2024-09-16T13:00:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034548018, 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036350421000000001, 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177007 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653999+0000", 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653999+0000", 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.653999+0000", 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654001+0000", 2024-09-16T13:00:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.345e-06 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654011+0000", 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6739999999999992e-06 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654015+0000", 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8399999999999997e-06 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177008 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654020+0000", 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034526955999999998, 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036405229999999997, 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177008 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654020+0000", 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654020+0000", 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654020+0000", 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.32e-07 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654023+0000", 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4559999999999999e-06 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654032+0000", 2024-09-16T13:00:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1470000000000007e-06 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654036+0000", 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0579999999999999e-06 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177009 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654040+0000", 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034506298999999997, 2024-09-16T13:00:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036438941000000002, 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177009 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654040+0000", 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654040+0000", 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654041+0000", 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654043+0000", 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.402e-06 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654053+0000", 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7189999999999996e-06 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654057+0000", 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8260000000000003e-06 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177010 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654061+0000", 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034485292000000001, 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036471374000000001, 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177010 2024-09-16T13:00:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654061+0000", 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654061+0000", 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654062+0000", 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1300000000000001e-07 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654064+0000", 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9309999999999998e-06 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654073+0000", 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2420000000000006e-06 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654077+0000", 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1099999999999996e-06 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177011 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654081+0000", 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034465425000000001, 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036505331000000002, 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177011 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654081+0000", 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654081+0000", 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654082+0000", 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2599999999999998e-07 2024-09-16T13:00:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654084+0000", 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1579999999999999e-06 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654090+0000", 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2489999999999998e-06 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654094+0000", 2024-09-16T13:00:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.912e-06 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:177012 4.f 4.4287356f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:29.654098+0000", 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.034448434999999999, 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036551818999999999, 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 177012 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654098+0000", 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654098+0000", 2024-09-16T13:00:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654099+0000", 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654100+0000", 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.784e-06 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654109+0000", 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8859999999999993e-06 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.654113+0000", 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.822e-06 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:00:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 22 2024-09-16T13:00:29.777 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:29.777 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:29.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:29.779 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:29.772+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:29.839 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /var/log/ceph/exp.3.3.4 2024-09-16T13:00:29.879 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:29.872+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:29.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T13:00:29.965 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:29.966 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:29.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:29.980 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:29.972+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:30.064 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:30.081 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:30.073+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:30.102 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:30.143 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:30.143 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:30.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:30.156 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:30.186 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:30.176+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:30.208 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:30.264 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:30.282 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:30.277+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:30.329 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:30.362 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:30.362 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:30.362 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:30.362 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:30.362 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:30.362 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:30.363 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:30.363 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:30.363 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:30.365 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:30.365 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:30.369 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:30.369 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:30.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:30.383 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:30.377+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:30.483 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:30.476+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:00:30.499 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":90,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T13:00:29.303250+0000","last_up_change":"2024-09-16T13:00:19.249119+0000","last_in_change":"2024-09-16T12:59:39.968761+0000","flags":"nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5804032,"flags_set":["nodeep-scrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T12:58:44.284177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T12:58:51.733915+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T12:59:00.193651+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T12:59:09.779825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":88,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":22186},{"type":"v1","addr":"172.21.15.42:6819","nonce":22186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":22186},{"type":"v1","addr":"172.21.15.42:6823","nonce":22186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":22186},{"type":"v1","addr":"172.21.15.42:6830","nonce":22186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":22186},{"type":"v1","addr":"172.21.15.42:6826","nonce":22186}]},"public_addr":"172.21.15.42:6819/22186","cluster_addr":"172.21.15.42:6823/22186","heartbeat_back_addr":"172.21.15.42:6830/22186","heartbeat_front_addr":"172.21.15.42:6826/22186","state":["exists","up"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":22184},{"type":"v1","addr":"172.21.15.42:6809","nonce":22184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":22184},{"type":"v1","addr":"172.21.15.42:6811","nonce":22184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":22184},{"type":"v1","addr":"172.21.15.42:6815","nonce":22184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":22184},{"type":"v1","addr":"172.21.15.42:6813","nonce":22184}]},"public_addr":"172.21.15.42:6809/22184","cluster_addr":"172.21.15.42:6811/22184","heartbeat_back_addr":"172.21.15.42:6815/22184","heartbeat_front_addr":"172.21.15.42:6813/22184","state":["exists","up"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":22185},{"type":"v1","addr":"172.21.15.42:6801","nonce":22185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":22185},{"type":"v1","addr":"172.21.15.42:6803","nonce":22185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":22185},{"type":"v1","addr":"172.21.15.42:6807","nonce":22185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":22185},{"type":"v1","addr":"172.21.15.42:6805","nonce":22185}]},"public_addr":"172.21.15.42:6801/22185","cluster_addr":"172.21.15.42:6803/22185","heartbeat_back_addr":"172.21.15.42:6807/22185","heartbeat_front_addr":"172.21.15.42:6805/22185","state":["exists","up"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":22187},{"type":"v1","addr":"172.21.15.42:6818","nonce":22187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":22187},{"type":"v1","addr":"172.21.15.42:6822","nonce":22187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":22187},{"type":"v1","addr":"172.21.15.42:6831","nonce":22187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":22187},{"type":"v1","addr":"172.21.15.42:6827","nonce":22187}]},"public_addr":"172.21.15.42:6818/22187","cluster_addr":"172.21.15.42:6822/22187","heartbeat_back_addr":"172.21.15.42:6831/22187","heartbeat_front_addr":"172.21.15.42:6827/22187","state":["exists","up"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22339},{"type":"v1","addr":"172.21.15.136:6817","nonce":22339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":22339},{"type":"v1","addr":"172.21.15.136:6819","nonce":22339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":22339},{"type":"v1","addr":"172.21.15.136:6823","nonce":22339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22339},{"type":"v1","addr":"172.21.15.136:6821","nonce":22339}]},"public_addr":"172.21.15.136:6817/22339","cluster_addr":"172.21.15.136:6819/22339","heartbeat_back_addr":"172.21.15.136:6823/22339","heartbeat_front_addr":"172.21.15.136:6821/22339","state":["exists"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22340},{"type":"v1","addr":"172.21.15.136:6809","nonce":22340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22340},{"type":"v1","addr":"172.21.15.136:6811","nonce":22340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22340},{"type":"v1","addr":"172.21.15.136:6815","nonce":22340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22340},{"type":"v1","addr":"172.21.15.136:6813","nonce":22340}]},"public_addr":"172.21.15.136:6809/22340","cluster_addr":"172.21.15.136:6811/22340","heartbeat_back_addr":"172.21.15.136:6815/22340","heartbeat_front_addr":"172.21.15.136:6813/22340","state":["exists","up"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22364},{"type":"v1","addr":"172.21.15.136:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":22364},{"type":"v1","addr":"172.21.15.136:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":22364},{"type":"v1","addr":"172.21.15.136:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22364},{"type":"v1","addr":"172.21.15.136:6829","nonce":22364}]},"public_addr":"172.21.15.136:6825/22364","cluster_addr":"172.21.15.136:6827/22364","heartbeat_back_addr":"172.21.15.136:6831/22364","heartbeat_front_addr":"172.21.15.136:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22341},{"type":"v1","addr":"172.21.15.136:6801","nonce":22341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22341},{"type":"v1","addr":"172.21.15.136:6803","nonce":22341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22341},{"type":"v1","addr":"172.21.15.136:6807","nonce":22341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22341},{"type":"v1","addr":"172.21.15.136:6805","nonce":22341}]},"public_addr":"172.21.15.136:6801/22341","cluster_addr":"172.21.15.136:6803/22341","heartbeat_back_addr":"172.21.15.136:6807/22341","heartbeat_front_addr":"172.21.15.136:6805/22341","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.264463+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.214474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.217927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.435822+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.451286+0000","dead_epoch":84},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.287403+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.440296+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.140630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":1,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T13:00:30.516 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:30.516 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:30.529 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:30.533 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T12:58:44.284177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T12:58:51.733915+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T12:59:00.193651+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T12:59:09.779825+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 88, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-09-16T13:00:30.533 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op apply-layout-settings --pool .mgr 2024-09-16T13:00:30.584 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:30.577+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:30.679 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:30.679 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:30.685 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:30.680+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:30.692 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:30.787 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:30.785+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:30.831 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:30.831 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:30.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:30.888 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:30.880+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:30.889 INFO:teuthology.orchestra.run.smithi042.stdout:4653 read 0x14c7111 thru 0x14c7213 (0x103 bytes) 2024-09-16T13:00:30.890 INFO:teuthology.orchestra.run.smithi042.stdout:4655 punch from 0x30b09a2 to 0x30b29ea, (0x2048 bytes) 2024-09-16T13:00:30.910 INFO:teuthology.orchestra.run.smithi042.stdout:4656 writesame 0xbe019a9 thru 0xbe0c3af data_size 0xb(0xaa07 bytes) 2024-09-16T13:00:30.917 INFO:teuthology.orchestra.run.smithi042.stdout:4658 trunc from 0xbe0c3b0 to 0x8417a20 2024-09-16T13:00:30.927 INFO:teuthology.orchestra.run.smithi042.stdout:4659 trunc from 0x8417a20 to 0x990cdbf 2024-09-16T13:00:30.935 INFO:teuthology.orchestra.run.smithi042.stdout:4660 read 0x86c993e thru 0x86d1266 (0x7929 bytes) 2024-09-16T13:00:30.935 INFO:teuthology.orchestra.run.smithi042.stdout:4661 write 0x6f3ea1c thru 0x6f461d9 (0x77be bytes) 2024-09-16T13:00:30.942 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:30.942 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:30.943 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:30.943 INFO:teuthology.orchestra.run.smithi042.stdout:4669 read 0x979eff7 thru 0x97a254b (0x3555 bytes) 2024-09-16T13:00:30.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:30.956 INFO:teuthology.orchestra.run.smithi042.stdout:4673 write 0xda57056 thru 0xda6108b (0xa036 bytes) 2024-09-16T13:00:30.962 INFO:teuthology.orchestra.run.smithi042.stdout:4674 punch from 0xb432da5 to 0xb4340f4, (0x134f bytes) 2024-09-16T13:00:30.962 INFO:teuthology.orchestra.run.smithi042.stdout:4675 write 0xcdbd920 thru 0xcdbe0f6 (0x7d7 bytes) 2024-09-16T13:00:30.965 INFO:teuthology.orchestra.run.smithi042.stdout:4677 read 0xb64a313 thru 0xb65072e (0x641c bytes) 2024-09-16T13:00:30.965 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:30.965 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:30.984 INFO:teuthology.orchestra.run.smithi042.stdout:4680 trunc from 0xda6108c to 0x12a2c46 2024-09-16T13:00:30.989 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:30.984+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:31.091 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:31.085+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:31.117 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:31.117 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:31.135 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:31.191 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:31.185+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:31.259 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:31.260 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:31.272 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:31.292 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:31.284+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:31.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T13:00:31.390 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:31.391 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:31.393 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:31.384+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:31.400 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:31.494 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:31.489+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:31.503 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:31.503 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:31.512 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:31.595 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:31.588+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:31.626 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:31.626 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:31.637 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:31.696 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:31.688+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2024-09-16T13:00:31.751 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:31.751 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:31.763 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:31.797 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:31.788+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:31.863 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:31.863 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:31.872 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:31.897 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:31.889+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:31.973 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:31.974 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:31.982 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:31.999 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:31.992+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:32.080 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:32.080 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:32.090 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:32.100 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:32.092+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:32.167 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:32.167 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:32.168 INFO:teuthology.orchestra.run.smithi042.stdout:4685 trunc from 0x12a2c46 to 0xc704e8 2024-09-16T13:00:32.182 INFO:teuthology.orchestra.run.smithi042.stdout:4686 read 0xc4566f thru 0xc49995 (0x4327 bytes) 2024-09-16T13:00:32.189 INFO:teuthology.orchestra.run.smithi042.stdout:4687 write 0x1228343 thru 0x122de20 (0x5ade bytes) 2024-09-16T13:00:32.192 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:32.192 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:32.192 INFO:teuthology.orchestra.run.smithi042.stdout:4688 read 0x429761 thru 0x43642d (0xcccd bytes) 2024-09-16T13:00:32.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:32.201 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:32.196+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:32.208 INFO:teuthology.orchestra.run.smithi042.stdout:4689 trunc from 0x122de21 to 0xada3774 2024-09-16T13:00:32.216 INFO:teuthology.orchestra.run.smithi042.stdout:4690 write 0xad8fff4 thru 0xad94119 (0x4126 bytes) 2024-09-16T13:00:32.224 INFO:teuthology.orchestra.run.smithi042.stdout:4691 read 0x3e0bcb1 thru 0x3e0d8d3 (0x1c23 bytes) 2024-09-16T13:00:32.224 INFO:teuthology.orchestra.run.smithi042.stdout:4692 read 0x30b44c3 thru 0x30b646c (0x1faa bytes) 2024-09-16T13:00:32.224 INFO:teuthology.orchestra.run.smithi042.stdout:4693 trunc from 0xada3774 to 0xb58253c 2024-09-16T13:00:32.232 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:32.232 INFO:teuthology.orchestra.run.smithi042.stdout:4697 writesame 0x410e5 thru 0x41b29 data_size 0x1(0xa45 bytes) 2024-09-16T13:00:32.235 INFO:teuthology.orchestra.run.smithi042.stdout:4699 read 0x227a83d thru 0x2281d7d (0x7541 bytes) 2024-09-16T13:00:32.250 INFO:teuthology.orchestra.run.smithi042.stdout:4700 trunc from 0xb58253c to 0x2552157 2024-09-16T13:00:32.269 INFO:teuthology.orchestra.run.smithi042.stdout:4704 read 0x1bcb5a thru 0x1c6cee (0xa195 bytes) 2024-09-16T13:00:32.270 INFO:teuthology.orchestra.run.smithi042.stdout:4705 read 0x673c44 thru 0x68090c (0xccc9 bytes) 2024-09-16T13:00:32.271 INFO:teuthology.orchestra.run.smithi042.stdout:4706 writesame 0x234f435 thru 0x235c81f data_size 0x1(0xd3eb bytes) 2024-09-16T13:00:32.289 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op apply-layout-settings --pool rbd 2024-09-16T13:00:32.301 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:32.293+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:32.303 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:32.303 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:32.305 INFO:teuthology.orchestra.run.smithi042.stdout:4708 punch from 0xe85059 to 0xe88e02, (0x3da9 bytes) 2024-09-16T13:00:32.305 INFO:teuthology.orchestra.run.smithi042.stdout:4709 punch from 0xdd9e55 to 0xde96f1, (0xf89c bytes) 2024-09-16T13:00:32.305 INFO:teuthology.orchestra.run.smithi042.stdout:4712 read 0x18586c7 thru 0x185bfd1 (0x390b bytes) 2024-09-16T13:00:32.305 INFO:teuthology.orchestra.run.smithi042.stdout:4713 punch from 0x1f30d15 to 0x1f34717, (0x3a02 bytes) 2024-09-16T13:00:32.305 INFO:teuthology.orchestra.run.smithi042.stdout:4714 read 0x13af7d0 thru 0x13b8c2e (0x945f bytes) 2024-09-16T13:00:32.306 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:32.314 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:32.318 INFO:teuthology.orchestra.run.smithi042.stdout:4717 write 0x77606ff thru 0x7762543 (0x1e45 bytes) 2024-09-16T13:00:32.323 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:32.338 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T13:00:32.341 INFO:teuthology.orchestra.run.smithi042.stdout:4720 writesame 0xa35c120 thru 0xa36469c data_size 0x3(0x857d bytes) 2024-09-16T13:00:32.341 INFO:teuthology.orchestra.run.smithi042.stdout:4722 punch from 0x81aa8e6 to 0x81b4fcf, (0xa6e9 bytes) 2024-09-16T13:00:32.341 INFO:teuthology.orchestra.run.smithi042.stdout:4724 read 0x57544a1 thru 0x57617c7 (0xd327 bytes) 2024-09-16T13:00:32.342 INFO:teuthology.orchestra.run.smithi042.stdout:4725 read 0x7795f4f thru 0x77a4d86 (0xee38 bytes) 2024-09-16T13:00:32.342 INFO:teuthology.orchestra.run.smithi042.stdout:4726 write 0x7ce35ea thru 0x7ce3a05 (0x41c bytes) 2024-09-16T13:00:32.357 INFO:teuthology.orchestra.run.smithi042.stdout:4727 trunc from 0xa36469d to 0xe06b49 2024-09-16T13:00:32.367 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:32.377 INFO:teuthology.orchestra.run.smithi042.stdout:4728 punch from 0x5ff36a to 0x5ff76d, (0x403 bytes) 2024-09-16T13:00:32.389 INFO:teuthology.orchestra.run.smithi042.stdout:4730 trunc from 0xe06b49 to 0x9c8dbb5 2024-09-16T13:00:32.396 INFO:teuthology.orchestra.run.smithi042.stdout:4733 read 0x984960 thru 0x989514 (0x4bb5 bytes) 2024-09-16T13:00:32.396 INFO:teuthology.orchestra.run.smithi042.stdout:4734 writesame 0x969e08f thru 0x969ed09 data_size 0xf(0xc7b bytes) 2024-09-16T13:00:32.400 INFO:teuthology.orchestra.run.smithi042.stdout:4736 read 0x3e2f113 thru 0x3e3bfbd (0xceab bytes) 2024-09-16T13:00:32.402 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:32.396+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:32.408 INFO:teuthology.orchestra.run.smithi042.stdout:4738 trunc from 0x9c8dbb5 to 0x30be252 2024-09-16T13:00:32.437 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:32.437 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:32.439 INFO:teuthology.orchestra.run.smithi042.stdout:4740 write 0xb74c131 thru 0xb7533e7 (0x72b7 bytes) 2024-09-16T13:00:32.446 INFO:teuthology.orchestra.run.smithi042.stdout:4741 write 0x6c8cba5 thru 0x6c8dcd7 (0x1133 bytes) 2024-09-16T13:00:32.449 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:32.453 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:32.453 INFO:teuthology.orchestra.run.smithi042.stdout:4744 read 0x33c349a thru 0x33c38d5 (0x43c bytes) 2024-09-16T13:00:32.453 INFO:teuthology.orchestra.run.smithi042.stdout:4745 writesame 0x244008e thru 0x2444133 data_size 0x19(0x40a6 bytes) 2024-09-16T13:00:32.456 INFO:teuthology.orchestra.run.smithi042.stdout:4746 writesame 0x1d2a345 thru 0x1d3362b data_size 0x1(0x92e7 bytes) 2024-09-16T13:00:32.482 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:32.482 INFO:teuthology.orchestra.run.smithi042.stdout:4749 clone 29 order 24 su 2097152 sc 10 2024-09-16T13:00:32.503 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:32.497+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:32.533 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851879369999999, 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851864343999999, 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851848321999999, 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851832225999999, 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851816335999999, 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851800665999999, 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851784919, 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851769131000001, 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851753809000002, 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851733884000001, 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851717936, 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851702775, 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851687127000002, 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.85167203, 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851656768000002, 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:00:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851641021999999, 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:00:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:00:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851625425000002, 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851610121, 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851594682999998, 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:00:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.851578969999998, 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:00:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:00:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:00:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:00:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.638 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:32.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:32.640 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:32.640 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:32.641 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:32.641 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:32.641 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:32.632+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:32.645 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:32.645 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:32.645 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:32.645 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:32.645 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:32.645 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:32.645 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:32.649 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:32.649 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:32.741 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:32.733+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:32.746 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:32.746 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:32.747 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:32.747 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:32.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:32.842 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:32.837+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:32.857 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:32.857 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:32.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:32.942 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:32.936+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:32.963 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:32.963 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:32.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:32.963 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166216 4.f 4:f0f74f87:::rbd_data.11b5c2ee3542.0000000000000003:1d [read 8261632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.954225+0000", 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0043062960000002, 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078866292000000005, 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166216 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.964 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954225+0000", 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954225+0000", 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954225+0000", 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.75e-07 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954227+0000", 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0650000000000001e-06 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954233+0000", 2024-09-16T13:00:32.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4990000000000002e-06 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954236+0000", 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3050000000000001e-06 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.032951+0000", 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078714471999999994 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033028+0000", 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7427000000000003e-05 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033091+0000", 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3048999999999999e-05 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166217 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8597504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:32.968 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.954240+0000", 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0042910770000004, 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079025498999999999, 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166217 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954240+0000", 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954240+0000", 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955245+0000", 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010046879999999999 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955250+0000", 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0969999999999996e-06 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955260+0000", 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0614e-05 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955266+0000", 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6910000000000002e-06 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033117+0000", 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077851358999999995 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033189+0000", 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1904999999999997e-05 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033265+0000", 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6144999999999997e-05 2024-09-16T13:00:32.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166224 4.f 4:f0f74f87:::rbd_data.11b5c2ee3542.0000000000000003:1d [read 8470528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955341+0000", 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0031904310000002, 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078746751000000004, 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166224 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955341+0000", 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955341+0000", 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955341+0000", 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7500000000000001e-07 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955343+0000", 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0880000000000002e-06 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955350+0000", 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5400000000000001e-06 2024-09-16T13:00:32.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955353+0000", 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.591e-06 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033952+0000", 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078598960999999995 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034028+0000", 2024-09-16T13:00:32.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5523000000000002e-05 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034087+0000", 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9673000000000003e-05 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166226 4.f 4:f0f74f87:::rbd_data.11b5c2ee3542.0000000000000003:1d [read 8515584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955357+0000", 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0031741109999999, 2024-09-16T13:00:32.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078887578999999999, 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166226 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955357+0000", 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955357+0000", 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955357+0000", 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955359+0000", 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955365+0000", 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7429999999999999e-06 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955368+0000", 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3400000000000002e-06 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034114+0000", 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078745599999999999 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034185+0000", 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1240000000000002e-05 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034244+0000", 2024-09-16T13:00:32.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9361000000000003e-05 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166228 4.f 4:f0f74f87:::rbd_data.11b5c2ee3542.0000000000000003:1d [read 8560640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955373+0000", 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0031583279999996, 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079032832999999997, 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166228 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955373+0000", 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955373+0000", 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955373+0000", 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955375+0000", 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955380+0000", 2024-09-16T13:00:32.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955384+0000", 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.625e-06 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034269+0000", 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078884923999999995 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034344+0000", 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4692999999999998e-05 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034406+0000", 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2003999999999998e-05 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166230 4.f 4:f0f74f87:::rbd_data.11b5c2ee3542.0000000000000003:1d [read 8605696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955388+0000", 2024-09-16T13:00:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.003143079, 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079187638000000005, 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166230 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955388+0000", 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955388+0000", 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955388+0000", 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0399999999999996e-07 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955390+0000", 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.979e-06 2024-09-16T13:00:32.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955396+0000", 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1529999999999996e-06 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955399+0000", 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.597e-06 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034432+0000", 2024-09-16T13:00:32.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079032701999999996 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034511+0000", 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8936000000000005e-05 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034576+0000", 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4766999999999998e-05 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166286 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8151040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955403+0000", 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0031278989999999, 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079434322000000002, 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166286 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.991 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955403+0000", 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955403+0000", 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955404+0000", 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8999999999999997e-07 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955406+0000", 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9039999999999999e-06 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955411+0000", 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8209999999999998e-06 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955415+0000", 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.546e-06 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034653+0000", 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079238266000000002 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034743+0000", 2024-09-16T13:00:32.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0199999999999997e-05 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034837+0000", 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4094999999999999e-05 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166288 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8196096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955419+0000", 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0031122679999998, 2024-09-16T13:00:32.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079603004000000005, 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166288 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955419+0000", 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955419+0000", 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955419+0000", 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9000000000000002e-07 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955425+0000", 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0739999999999998e-06 2024-09-16T13:00:32.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955432+0000", 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.754e-06 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955445+0000", 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2969999999999999e-05 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:32.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034866+0000", 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079420705999999994 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034938+0000", 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2479999999999997e-05 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035022+0000", 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3629999999999997e-05 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:32.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166289 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8241152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955450+0000", 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0030810890000001, 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079755874000000004, 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166289 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955450+0000", 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955450+0000", 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955450+0000", 2024-09-16T13:00:33.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955453+0000", 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2570000000000002e-06 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955459+0000", 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1700000000000002e-06 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955463+0000", 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8820000000000003e-06 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035048+0000", 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079585521000000006 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035122+0000", 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3971e-05 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035206+0000", 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3607000000000001e-05 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166290 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8286208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955467+0000", 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0030635529999996, 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079938254, 2024-09-16T13:00:33.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166290 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955467+0000", 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955467+0000", 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955468+0000", 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2500000000000001e-07 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955470+0000", 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9460000000000001e-06 2024-09-16T13:00:33.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955475+0000", 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6500000000000001e-06 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955479+0000", 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6339999999999999e-06 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035238+0000", 2024-09-16T13:00:33.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079758684999999996 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035319+0000", 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1392999999999998e-05 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035406+0000", 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6520999999999994e-05 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166291 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8331264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955483+0000", 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0030478110000001, 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080127871000000003, 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166291 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955483+0000", 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955483+0000", 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955536+0000", 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2454000000000003e-05 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955539+0000", 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5420000000000001e-06 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955548+0000", 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.32e-06 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955552+0000", 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.51e-06 2024-09-16T13:00:33.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035446+0000", 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079893756999999996 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035526+0000", 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0517000000000002e-05 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035611+0000", 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4771000000000005e-05 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166292 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8376320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955557+0000", 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0029737360000004, 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080247081999999997, 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166292 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955557+0000", 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955557+0000", 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955771+0000", 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021362900000000001 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955776+0000", 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1340000000000003e-06 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955847+0000", 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0865000000000007e-05 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955860+0000", 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2951e-05 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035644+0000", 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079784069999999999 2024-09-16T13:00:33.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035721+0000", 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6763999999999999e-05 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035804+0000", 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3669000000000001e-05 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166293 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8404992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955867+0000", 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0026637579999997, 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080124405999999995, 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166293 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955867+0000", 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955867+0000", 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956289+0000", 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042214200000000003 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956294+0000", 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4519999999999999e-06 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956307+0000", 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2802999999999999e-05 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956312+0000", 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5489999999999997e-06 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035831+0000", 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079518988999999998 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035906+0000", 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4995999999999998e-05 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035992+0000", 2024-09-16T13:00:33.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5475000000000004e-05 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166295 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8450048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.956317+0000", 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0022141299999996, 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079894277, 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.022 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166295 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956317+0000", 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956317+0000", 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957407+0000", 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010904319999999999 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957412+0000", 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1710000000000001e-06 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957872+0000", 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045938700000000001 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957880+0000", 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5359999999999992e-06 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036026+0000", 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078145237000000006 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036116+0000", 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0354999999999998e-05 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036211+0000", 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5159000000000002e-05 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166300 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8585216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.957930+0000", 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0006011380000004, 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920253999999995, 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166300 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957930+0000", 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957930+0000", 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957930+0000", 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6699999999999999e-07 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957932+0000", 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1289999999999999e-06 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957938+0000", 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0229999999999998e-06 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957942+0000", 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8039999999999999e-06 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036665+0000", 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078722996000000003 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036756+0000", 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0742999999999995e-05 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036850+0000", 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4092000000000006e-05 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166303 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8630272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.957947+0000", 2024-09-16T13:00:33.030 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0005845019999997, 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079099346000000001, 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166303 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957947+0000", 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957947+0000", 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957947+0000", 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957949+0000", 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0889999999999998e-06 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957954+0000", 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7559999999999999e-06 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957956+0000", 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.154e-06 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036877+0000", 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920855999999998 2024-09-16T13:00:33.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036963+0000", 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6297000000000004e-05 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.037046+0000", 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2791000000000001e-05 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166494 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 7073792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.963734+0000", 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.9947967260000001, 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078831144000000006, 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166494 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963734+0000", 2024-09-16T13:00:33.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963734+0000", 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963735+0000", 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6300000000000001e-07 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963736+0000", 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.748e-06 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963742+0000", 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.772e-06 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963746+0000", 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.608e-06 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042376+0000", 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078629839000000007 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042467+0000", 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1197000000000001e-05 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042565+0000", 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8616999999999994e-05 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166496 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 7102464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.963750+0000", 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.9947813229999998, 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079033872000000005, 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166496 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.039 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963750+0000", 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963750+0000", 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963750+0000", 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5599999999999995e-07 2024-09-16T13:00:33.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963752+0000", 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8059999999999999e-06 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963758+0000", 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7189999999999998e-06 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963761+0000", 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6509999999999999e-06 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042598+0000", 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078836421000000004 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042686+0000", 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8010999999999995e-05 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042784+0000", 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7707999999999994e-05 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166498 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 7147520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.963781+0000", 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.9947497080000001, 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079226252999999996, 2024-09-16T13:00:33.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166498 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963781+0000", 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963781+0000", 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963782+0000", 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963784+0000", 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.187e-06 2024-09-16T13:00:33.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963791+0000", 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9820000000000002e-06 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963795+0000", 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1420000000000003e-06 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042814+0000", 2024-09-16T13:00:33.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079018599999999994 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042910+0000", 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6229000000000003e-05 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.043008+0000", 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7695000000000006e-05 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166499 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 7192576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.963799+0000", 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.994731877, 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079436572999999996, 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166499 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963799+0000", 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963799+0000", 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.973831+0000", 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010031722999999999 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.973838+0000", 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3880000000000002e-06 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.979866+0000", 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060278850000000002 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.979881+0000", 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4576000000000001e-05 2024-09-16T13:00:33.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.043038+0000", 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.063157074999999993 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.043128+0000", 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9901000000000005e-05 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.043236+0000", 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108025 2024-09-16T13:00:33.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.052 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.052 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.052 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:33.053 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:33.054 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:33.048+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:33.155 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:33.149+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:33.209 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:33.255 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:33.249+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:33.309 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:33.309 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:33.309 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:33.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:33.356 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:33.348+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:33.429 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:33.429 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:33.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:33.429 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54223 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.768302890000001, 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028808500000000001, 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54223 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655799+0000", 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:33.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655801+0000", 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6579999999999998e-06 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655811+0000", 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5070000000000006e-06 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655816+0000", 2024-09-16T13:00:33.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8960000000000002e-06 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684450+0000", 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028634057000000001 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684521+0000", 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1453999999999997e-05 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684607+0000", 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5447999999999999e-05 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54224 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1220608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.768280088000001, 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028979718000000002, 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54224 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:33.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655822+0000", 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2600000000000002e-07 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655824+0000", 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.593e-06 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655833+0000", 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2099999999999999e-06 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655838+0000", 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5650000000000003e-06 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684637+0000", 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028798550999999999 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684715+0000", 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8242000000000001e-05 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684801+0000", 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6031000000000004e-05 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.438 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54225 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.768257971000001, 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029147567999999999, 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54225 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:33.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655844+0000", 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.06e-07 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655846+0000", 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5059999999999998e-06 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655856+0000", 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.825e-06 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655861+0000", 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6990000000000002e-06 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684830+0000", 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028969385 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684907+0000", 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6800999999999999e-05 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684991+0000", 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3845999999999996e-05 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.442 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54226 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.768235031, 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029452944000000002, 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54226 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.443 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655867+0000", 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8599999999999998e-07 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655869+0000", 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5849999999999998e-06 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655878+0000", 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2620000000000004e-06 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655883+0000", 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7350000000000004e-06 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685017+0000", 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029134083000000002 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685096+0000", 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999994e-05 2024-09-16T13:00:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685319+0000", 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022352299999999999 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54227 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1318912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.768212544000001, 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029655159, 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54227 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5700000000000002e-07 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655892+0000", 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.486e-06 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655903+0000", 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1e-05 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655907+0000", 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.831e-06 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685347+0000", 2024-09-16T13:00:33.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029439844 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685416+0000", 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8561999999999998e-05 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685544+0000", 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012787900000000001 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54228 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.768188199000001, 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029867089999999999, 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54228 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.452 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2499999999999995e-07 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655916+0000", 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9170000000000002e-06 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655926+0000", 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7769999999999997e-06 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655931+0000", 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8470000000000003e-06 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685571+0000", 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029640061999999998 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685642+0000", 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1327000000000003e-05 2024-09-16T13:00:33.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685780+0000", 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013763499999999999 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54229 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.768164492, 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030082340999999999, 2024-09-16T13:00:33.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54229 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1500000000000005e-07 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655940+0000", 2024-09-16T13:00:33.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5840000000000002e-06 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655949+0000", 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9979999999999992e-06 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655954+0000", 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.746e-06 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685808+0000", 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029854776999999999 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685879+0000", 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0593000000000007e-05 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.686019+0000", 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000140128 2024-09-16T13:00:33.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54832 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 192512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.721511327000002, 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.032379022, 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54832 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4099999999999999e-07 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702592+0000", 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.995e-06 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702597+0000", 2024-09-16T13:00:33.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2120000000000002e-06 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702600+0000", 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9919999999999999e-06 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.727494+0000", 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024893141000000001 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.727564+0000", 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0761999999999998e-05 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734969+0000", 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0074044790000000003 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54858 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1044480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.720192947000001, 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028959381999999999, 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54858 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703909+0000", 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7599999999999997e-07 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703911+0000", 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4789999999999999e-06 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703917+0000", 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2389999999999999e-06 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703920+0000", 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.021e-06 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732697+0000", 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028776665 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732778+0000", 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1018000000000003e-05 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732868+0000", 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9483999999999999e-05 2024-09-16T13:00:33.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54859 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1077248~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.720175512000001, 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029138931999999999, 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54859 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9400000000000001e-07 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703928+0000", 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1179999999999999e-06 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703934+0000", 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.84e-06 2024-09-16T13:00:33.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703938+0000", 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5219999999999999e-06 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732893+0000", 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028955005999999998 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732974+0000", 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1569000000000005e-05 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733065+0000", 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0482999999999994e-05 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54860 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1110016~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.474 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.720159432999999, 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029318429999999999, 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54860 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:33.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703944+0000", 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9209999999999999e-06 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704020+0000", 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6217999999999994e-05 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704027+0000", 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3559999999999998e-06 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733091+0000", 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029064349999999999 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733167+0000", 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6058999999999998e-05 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733260+0000", 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3110999999999997e-05 2024-09-16T13:00:33.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54861 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.720068242, 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029431589000000001, 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54861 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704034+0000", 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6199999999999997e-07 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704037+0000", 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9100000000000001e-06 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704045+0000", 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2150000000000001e-06 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704048+0000", 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6380000000000002e-06 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733285+0000", 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029236999999999999 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733365+0000", 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9461000000000004e-05 2024-09-16T13:00:33.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733465+0000", 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9603000000000002e-05 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54862 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1175552~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.720048569999999, 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029595202000000001, 2024-09-16T13:00:33.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54862 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3300000000000002e-07 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704055+0000", 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0940000000000002e-06 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704061+0000", 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.835e-06 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704064+0000", 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.948e-06 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733485+0000", 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029420565999999999 2024-09-16T13:00:33.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733552+0000", 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7409999999999993e-05 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733648+0000", 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5816000000000006e-05 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54863 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1208320~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.720033772000001, 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029759681, 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54863 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704068+0000", 2024-09-16T13:00:33.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704076+0000", 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0299999999999994e-06 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704082+0000", 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.755e-06 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704085+0000", 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1159999999999999e-06 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733670+0000", 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029585163000000001 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733742+0000", 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2069000000000004e-05 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733827+0000", 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5161000000000007e-05 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54864 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.720012648000001, 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927993, 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54864 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3599999999999999e-07 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704091+0000", 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-06 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704096+0000", 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0950000000000003e-06 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704099+0000", 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8329999999999998e-06 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733853+0000", 2024-09-16T13:00:33.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029754368999999999 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733931+0000", 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7956999999999999e-05 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734017+0000", 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5272999999999995e-05 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54865 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1273856~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.719998714999999, 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030108264999999999, 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54865 2024-09-16T13:00:33.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704103+0000", 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704104+0000", 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.716e-06 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704110+0000", 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4410000000000001e-06 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704113+0000", 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1839999999999999e-06 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734040+0000", 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927013999999998 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734122+0000", 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1481000000000001e-05 2024-09-16T13:00:33.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734211+0000", 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9140000000000004e-05 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54866 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1306624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:33.501 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.719984304, 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030274079999999998, 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54866 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704119+0000", 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9870000000000002e-06 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704125+0000", 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3410000000000002e-06 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704133+0000", 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734225+0000", 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030092026000000001 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734306+0000", 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0634000000000001e-05 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734391+0000", 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5082999999999998e-05 2024-09-16T13:00:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54867 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.719964230999999, 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030392758999999998, 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54867 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704139+0000", 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.99e-06 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704145+0000", 2024-09-16T13:00:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704148+0000", 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.078e-06 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734412+0000", 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030264287000000001 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734469+0000", 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6925999999999997e-05 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734530+0000", 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0381999999999999e-05 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54868 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1372160~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.719949536000001, 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030530140000000001, 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54868 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704245+0000", 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3037999999999999e-05 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704256+0000", 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0718e-05 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704261+0000", 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734544+0000", 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030283929000000001 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734595+0000", 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0356999999999999e-05 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734682+0000", 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6938e-05 2024-09-16T13:00:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54869 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1404928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.719835295999999, 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030584762000000001, 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54869 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704267+0000", 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4700000000000001e-07 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704269+0000", 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1390000000000002e-06 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704275+0000", 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9449999999999998e-06 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704278+0000", 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1429999999999998e-06 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734705+0000", 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030427551000000001 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734778+0000", 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2502000000000005e-05 2024-09-16T13:00:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734851+0000", 2024-09-16T13:00:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2835000000000003e-05 2024-09-16T13:00:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:33.519 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:33.520 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2024-09-16T13:00:33.521 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:33.521 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone28 from 0xb7533e8 (overlap 0xc704e8) to 0x3bb4ac 2024-09-16T13:00:33.521 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:33.512+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:33.621 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:33.613+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:33.682 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:33.682 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:33.683 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:33.683 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:33.687 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-16T13:00:33.695 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:33.723 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:33.716+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:33.822 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:33.816+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:33.849 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:33.849 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:33.849 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:33.862 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:33.926 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:33.916+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:33.999 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op apply-layout-settings --pool cache 2024-09-16T13:00:34.042 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:34.037+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:34.052 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:34.052 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:34.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:34.052 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:34.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.052 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:34.052 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.9786113059999995, 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:00:34.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.9785588870000002, 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:00:34.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:34.061 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.9785420390000006, 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:34.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:00:34.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.9785259249999996, 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:00:34.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:34.070 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.9784609950000007, 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:00:34.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.9784448959999992, 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:00:34.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:34.079 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.9783711149999998, 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:34.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:00:34.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.9783539139999995, 2024-09-16T13:00:34.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.9783372109999995, 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:00:34.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:00:34.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4420259560000002, 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:34.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4420088199999999, 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:00:34.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:00:34.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4419892379999997, 2024-09-16T13:00:34.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:00:34.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:00:34.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4419583789999999, 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:00:34.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4418902830000002, 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:00:34.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159502 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4493312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612957+0000", 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4346025930000001, 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070340508999999996, 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159502 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612957+0000", 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612957+0000", 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612958+0000", 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612960+0000", 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3750000000000001e-06 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612967+0000", 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3929999999999996e-06 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612970+0000", 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.23e-06 2024-09-16T13:00:34.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683111+0000", 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070141677999999999 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683188+0000", 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6157999999999998e-05 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683298+0000", 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110221 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4345864280000002, 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4345688389999998, 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:00:34.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4345539460000003, 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:34.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.434522329, 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:00:34.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:00:34.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4345043080000002, 2024-09-16T13:00:34.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:00:34.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:00:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.141 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:34.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:34.143 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:34.143 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:34.137+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:34.244 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:34.244 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:34.244 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:34.244 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:34.244 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:34.237+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:34.253 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:34.323 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:34.323 INFO:teuthology.orchestra.run.smithi042.stdout:4751 write 0x57ba9e0 thru 0x57bae2a (0x44b bytes) 2024-09-16T13:00:34.344 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:34.336+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:34.347 INFO:teuthology.orchestra.run.smithi042.stdout:4752 punch from 0x8fe9079 to 0x8fe9997, (0x91e bytes) 2024-09-16T13:00:34.347 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:34.350 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T13:00:34.351 INFO:teuthology.orchestra.run.smithi042.stdout:4754 trunc from 0xb7533e8 to 0xdf0d10a 2024-09-16T13:00:34.359 INFO:teuthology.orchestra.run.smithi042.stdout:4755 read 0xc2e5619 thru 0xc2e6126 (0xb0e bytes) 2024-09-16T13:00:34.359 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:34.360 INFO:teuthology.orchestra.run.smithi042.stdout:4757 punch from 0x562b5e4 to 0x5632d9b, (0x77b7 bytes) 2024-09-16T13:00:34.360 INFO:teuthology.orchestra.run.smithi042.stdout:4758 write 0xdde4527 thru 0xddf422f (0xfd09 bytes) 2024-09-16T13:00:34.365 INFO:teuthology.orchestra.run.smithi042.stdout:4759 writesame 0x80be17d thru 0x80c526e data_size 0x4f(0x70f2 bytes) 2024-09-16T13:00:34.377 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:34.378 INFO:teuthology.orchestra.run.smithi042.stdout:4762 write 0x7649c1 thru 0x76c0b8 (0x76f8 bytes) 2024-09-16T13:00:34.381 INFO:teuthology.orchestra.run.smithi042.stdout:4763 read 0x2741d1 thru 0x277f16 (0x3d46 bytes) 2024-09-16T13:00:34.397 INFO:teuthology.orchestra.run.smithi042.stdout:4765 trunc from 0xdf0d10a to 0x34d5dd1 2024-09-16T13:00:34.405 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:34.405 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:34.420 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:34.444 INFO:teuthology.orchestra.run.smithi042.stdout:4766 read 0x2091457 thru 0x20a0fe2 (0xfb8c bytes) 2024-09-16T13:00:34.445 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:34.437+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:34.452 INFO:teuthology.orchestra.run.smithi042.stdout:4767 trunc from 0x34d5dd1 to 0x83a6f32 2024-09-16T13:00:34.471 INFO:teuthology.orchestra.run.smithi042.stdout:4768 write 0xca97708 thru 0xcaa55b8 (0xdeb1 bytes) 2024-09-16T13:00:34.476 INFO:teuthology.orchestra.run.smithi042.stdout:4770 read 0xab2030c thru 0xab21905 (0x15fa bytes) 2024-09-16T13:00:34.476 INFO:teuthology.orchestra.run.smithi042.stdout:4772 read 0xc0a6e18 thru 0xc0b11ff (0xa3e8 bytes) 2024-09-16T13:00:34.476 INFO:teuthology.orchestra.run.smithi042.stdout:4774 writesame 0x7553448 thru 0x7561d2d data_size 0x3(0xe8e6 bytes) 2024-09-16T13:00:34.487 INFO:teuthology.orchestra.run.smithi042.stdout:4775 read 0x65c1afc thru 0x65d08f7 (0xedfc bytes) 2024-09-16T13:00:34.488 INFO:teuthology.orchestra.run.smithi042.stdout:4776 read 0x6baaead thru 0x6bac9bb (0x1b0f bytes) 2024-09-16T13:00:34.489 INFO:teuthology.orchestra.run.smithi042.stdout:4781 writesame 0x49e070a thru 0x49e44e5 data_size 0x4(0x3ddc bytes) 2024-09-16T13:00:34.494 INFO:teuthology.orchestra.run.smithi042.stdout:4782 read 0x587b52e thru 0x587b7c8 (0x29b bytes) 2024-09-16T13:00:34.495 INFO:teuthology.orchestra.run.smithi042.stdout:4783 write 0x9c3a390 thru 0x9c47f03 (0xdb74 bytes) 2024-09-16T13:00:34.499 INFO:teuthology.orchestra.run.smithi042.stdout:4784 read 0x1b86c66 thru 0x1b898de (0x2c79 bytes) 2024-09-16T13:00:34.499 INFO:teuthology.orchestra.run.smithi042.stdout:4785 writesame 0x4ccaf5e thru 0x4ccd40f data_size 0x16(0x24b2 bytes) 2024-09-16T13:00:34.503 INFO:teuthology.orchestra.run.smithi042.stdout:4786 write 0x48b0cab thru 0x48b4d1f (0x4075 bytes) 2024-09-16T13:00:34.505 INFO:teuthology.orchestra.run.smithi042.stdout:4789 read 0x873429b thru 0x873d745 (0x94ab bytes) 2024-09-16T13:00:34.505 INFO:teuthology.orchestra.run.smithi042.stdout:4792 punch from 0x92e4f15 to 0x92f03b7, (0xb4a2 bytes) 2024-09-16T13:00:34.505 INFO:teuthology.orchestra.run.smithi042.stdout:4793 write 0x8a6cf43 thru 0x8a79ddb (0xce99 bytes) 2024-09-16T13:00:34.507 INFO:teuthology.orchestra.run.smithi042.stdout:4794 write 0x13ec31a thru 0x13f70b7 (0xad9e bytes) 2024-09-16T13:00:34.510 INFO:teuthology.orchestra.run.smithi042.stdout:4795 punch from 0x8358dee to 0x835f613, (0x6825 bytes) 2024-09-16T13:00:34.510 INFO:teuthology.orchestra.run.smithi042.stdout:4796 write 0x8f47206 thru 0x8f4d990 (0x678b bytes) 2024-09-16T13:00:34.529 INFO:teuthology.orchestra.run.smithi042.stdout:4797 trunc from 0xcaa55b9 to 0x8b3e44 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8419 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.132589105999998, 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040293592000000003, 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.540 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8419 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402894+0000", 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402907+0000", 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402940+0000", 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3436000000000003e-05 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402953+0000", 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2840000000000001e-05 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403104+0000", 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150712 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403202+0000", 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8403e-05 2024-09-16T13:00:34.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.411015+0000", 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078133079999999997 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443018+0000", 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032002348999999999 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443075+0000", 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7703e-05 2024-09-16T13:00:34.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443141+0000", 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5705999999999997e-05 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443185+0000", 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4152e-05 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569356748000001, 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039945895000000002, 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48384 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966126+0000", 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1110000000000002e-06 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966134+0000", 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9740000000000002e-06 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966139+0000", 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4270000000000001e-06 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.005829+0000", 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039690007999999999 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.005928+0000", 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9184000000000004e-05 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006070+0000", 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014171100000000001 2024-09-16T13:00:34.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569337117, 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040120731, 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.550 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48385 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.63e-07 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966146+0000", 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.018e-06 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966154+0000", 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9999999999999996e-06 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966169+0000", 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5395000000000001e-05 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006118+0000", 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039948637000000002 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006181+0000", 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3001000000000002e-05 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006264+0000", 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3217e-05 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569305994, 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040378561, 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.554 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48386 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5199999999999996e-07 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966178+0000", 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.373e-06 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966186+0000", 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3259999999999996e-06 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966202+0000", 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6333000000000001e-05 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006311+0000", 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040108447999999998 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006411+0000", 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100141 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006553+0000", 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014218799999999999 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:34.558 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569272463000001, 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040537542000000003, 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48387 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966209+0000", 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4300000000000003e-07 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966211+0000", 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.418e-06 2024-09-16T13:00:34.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966219+0000", 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.627e-06 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966223+0000", 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3019999999999996e-06 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006601+0000", 2024-09-16T13:00:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040377953000000001 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006665+0000", 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3592999999999999e-05 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006746+0000", 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1106000000000006e-05 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569252864999999, 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040793276000000003, 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48388 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:34.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5700000000000002e-07 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966230+0000", 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0499999999999999e-06 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966246+0000", 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5543e-05 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966250+0000", 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0840000000000002e-06 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006794+0000", 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040543691999999999 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006901+0000", 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107754 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007021+0000", 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119596 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569225464999999, 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040958857000000001, 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48389 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966256+0000", 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8100000000000003e-07 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966258+0000", 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3829999999999999e-06 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966268+0000", 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8379999999999997e-06 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966272+0000", 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9940000000000002e-06 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007069+0000", 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040796897999999998 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007132+0000", 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3084999999999997e-05 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007214+0000", 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2177999999999998e-05 2024-09-16T13:00:34.570 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569202987000001, 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041219483000000001, 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48390 2024-09-16T13:00:34.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4700000000000002e-07 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966280+0000", 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3e-06 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966286+0000", 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.57e-06 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966288+0000", 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4439999999999998e-06 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007262+0000", 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040973242 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007351+0000", 2024-09-16T13:00:34.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9339999999999995e-05 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007497+0000", 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014614000000000001 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.575 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569181323999999, 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041394590000000002, 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48391 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966300+0000", 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966301+0000", 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5510000000000001e-06 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966305+0000", 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1849999999999997e-06 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966310+0000", 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.211e-06 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007546+0000", 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041236341000000003 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007610+0000", 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3534999999999994e-05 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007694+0000", 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4407999999999996e-05 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48392 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 794624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.56916867, 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041645095, 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48392 2024-09-16T13:00:34.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:34.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3599999999999999e-07 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966314+0000", 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7689999999999999e-06 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966322+0000", 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9340000000000006e-06 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966325+0000", 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5579999999999999e-06 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007742+0000", 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041417127999999998 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007840+0000", 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8199e-05 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007957+0000", 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011717099999999999 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48393 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569153474, 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041849738999999997, 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.584 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48393 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:34.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966328+0000", 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966340+0000", 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2604e-05 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966344+0000", 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1989999999999999e-06 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966348+0000", 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4039999999999999e-06 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008030+0000", 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041682036999999998 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008093+0000", 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3511999999999998e-05 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008177+0000", 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3561999999999997e-05 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48394 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:34.588 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569130014999999, 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042095443000000003, 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48394 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966352+0000", 2024-09-16T13:00:34.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.406e-06 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966356+0000", 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966362+0000", 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2299999999999996e-06 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008232+0000", 2024-09-16T13:00:34.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041869361000000001 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008320+0000", 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8492000000000006e-05 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008446+0000", 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012591800000000001 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48395 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569114473999999, 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042265693999999999, 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48395 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966367+0000", 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966368+0000", 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9420000000000002e-06 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966372+0000", 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9049999999999999e-06 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966375+0000", 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4329999999999998e-06 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008483+0000", 2024-09-16T13:00:34.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042108244000000003 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008546+0000", 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2973999999999997e-05 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008632+0000", 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.585e-05 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48396 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569103386999998, 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042505605000000002, 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48396 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966378+0000", 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966379+0000", 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.147e-06 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966382+0000", 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966385+0000", 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.43e-06 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008677+0000", 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042292191 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008765+0000", 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8027000000000003e-05 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008883+0000", 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117817 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.600 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48397 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569093480999999, 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042627777999999998, 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48397 2024-09-16T13:00:34.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966388+0000", 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966389+0000", 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.144e-06 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966392+0000", 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5939999999999998e-06 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966398+0000", 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3959999999999997e-06 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008904+0000", 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042505730999999998 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008953+0000", 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9786000000000001e-05 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009015+0000", 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1580000000000003e-05 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48398 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569080139, 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043115957000000003, 2024-09-16T13:00:34.605 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48398 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:34.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2500000000000001e-07 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966402+0000", 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3120000000000001e-06 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966406+0000", 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.653e-06 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966408+0000", 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.322e-06 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009095+0000", 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042687260999999997 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009276+0000", 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000180345 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009516+0000", 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024073899999999999 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48399 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:34.609 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569069406000001, 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043478389999999999, 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48399 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966412+0000", 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.03e-07 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966413+0000", 2024-09-16T13:00:34.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2410000000000001e-06 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966416+0000", 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6330000000000002e-06 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966422+0000", 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2379999999999997e-06 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009539+0000", 2024-09-16T13:00:34.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043116865999999997 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009588+0000", 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9731999999999998e-05 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009890+0000", 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030137700000000003 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.613 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48400 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569055676000001, 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043788502, 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48400 2024-09-16T13:00:34.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:34.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3999999999999997e-07 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966427+0000", 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.765e-06 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966434+0000", 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6989999999999997e-06 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966437+0000", 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1109999999999999e-06 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009969+0000", 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043532556 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010101+0000", 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0001317 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010213+0000", 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011233100000000001 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48401 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569041064, 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043965861000000002, 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48401 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966442+0000", 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8300000000000001e-06 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966455+0000", 2024-09-16T13:00:34.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2927e-05 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966459+0000", 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3850000000000004e-06 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010261+0000", 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043801305999999998 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010335+0000", 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4160999999999997e-05 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010405+0000", 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0716000000000005e-05 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.622 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48402 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.569016565999998, 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044179172000000003, 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48402 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969465+0000", 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003000689 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969469+0000", 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5739999999999997e-06 2024-09-16T13:00:34.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969483+0000", 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3692e-05 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969488+0000", 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6450000000000004e-06 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010444+0000", 2024-09-16T13:00:34.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040956030999999997 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010542+0000", 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8486000000000005e-05 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010643+0000", 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101055 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:34.626 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:34.627 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:34.628 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:34.629 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:34.629 INFO:teuthology.orchestra.run.smithi042.stdout:4800 write 0x4e63f08 thru 0x4e73d85 (0xfe7e bytes) 2024-09-16T13:00:34.629 INFO:teuthology.orchestra.run.smithi042.stdout:4801 read 0x4470839 thru 0x447e6e3 (0xdeab bytes) 2024-09-16T13:00:34.629 INFO:teuthology.orchestra.run.smithi042.stdout:4802 read 0x2ac5ebe thru 0x2ac6c0f (0xd52 bytes) 2024-09-16T13:00:34.629 INFO:teuthology.orchestra.run.smithi042.stdout:4803 write 0x39db78a thru 0x39e3b36 (0x83ad bytes) 2024-09-16T13:00:34.629 INFO:teuthology.orchestra.run.smithi042.stdout:4805 write 0xba252c4 thru 0xba2e4f2 (0x922f bytes) 2024-09-16T13:00:34.629 INFO:teuthology.orchestra.run.smithi042.stdout:4806 writesame 0xe199dbc thru 0xe1a111f data_size 0x23(0x7364 bytes) 2024-09-16T13:00:34.630 INFO:teuthology.orchestra.run.smithi042.stdout:4807 trunc from 0xe1a1120 to 0xbe19ee7 2024-09-16T13:00:34.630 INFO:teuthology.orchestra.run.smithi042.stdout:4808 write 0x691c98 thru 0x6956d7 (0x3a40 bytes) 2024-09-16T13:00:34.630 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:34.620+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:34.631 INFO:teuthology.orchestra.run.smithi042.stdout:4809 punch from 0x75c6699 to 0x75d41d3, (0xdb3a bytes) 2024-09-16T13:00:34.631 INFO:teuthology.orchestra.run.smithi042.stdout:4810 write 0x806800 thru 0x80e11d (0x791e bytes) 2024-09-16T13:00:34.634 INFO:teuthology.orchestra.run.smithi042.stdout:4811 write 0x270fc20 thru 0x2719907 (0x9ce8 bytes) 2024-09-16T13:00:34.635 INFO:teuthology.orchestra.run.smithi042.stdout:4812 read 0x531c081 thru 0x53231ac (0x712c bytes) 2024-09-16T13:00:34.637 INFO:teuthology.orchestra.run.smithi042.stdout:4815 punch from 0x6e9afd1 to 0x6ea0bca, (0x5bf9 bytes) 2024-09-16T13:00:34.637 INFO:teuthology.orchestra.run.smithi042.stdout:4816 write 0x34a586c thru 0x34b11b4 (0xb949 bytes) 2024-09-16T13:00:34.642 INFO:teuthology.orchestra.run.smithi042.stdout:4818 read 0x711a169 thru 0x712412a (0x9fc2 bytes) 2024-09-16T13:00:34.642 INFO:teuthology.orchestra.run.smithi042.stdout:4820 read 0x9e0871c thru 0x9e1405e (0xb943 bytes) 2024-09-16T13:00:34.643 INFO:teuthology.orchestra.run.smithi042.stdout:4821 read 0x7da61eb thru 0x7da9844 (0x365a bytes) 2024-09-16T13:00:34.643 INFO:teuthology.orchestra.run.smithi042.stdout:4822 read 0x565b800 thru 0x566854a (0xcd4b bytes) 2024-09-16T13:00:34.644 INFO:teuthology.orchestra.run.smithi042.stdout:4823 read 0x9d889b7 thru 0x9d8c59a (0x3be4 bytes) 2024-09-16T13:00:34.644 INFO:teuthology.orchestra.run.smithi042.stdout:4825 writesame 0xb0b29ee thru 0xb0baa1a data_size 0x13(0x802d bytes) 2024-09-16T13:00:34.650 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:34.651 INFO:teuthology.orchestra.run.smithi042.stdout:4827 read 0xb6ef0c3 thru 0xb6f884f (0x978d bytes) 2024-09-16T13:00:34.651 INFO:teuthology.orchestra.run.smithi042.stdout:4828 read 0xb661e08 thru 0xb667baf (0x5da8 bytes) 2024-09-16T13:00:34.651 INFO:teuthology.orchestra.run.smithi042.stdout:4829 write 0x3196659 thru 0x31a646a (0xfe12 bytes) 2024-09-16T13:00:34.654 INFO:teuthology.orchestra.run.smithi042.stdout:4830 write 0x87a3457 thru 0x87a7d21 (0x48cb bytes) 2024-09-16T13:00:34.661 INFO:teuthology.orchestra.run.smithi042.stdout:4832 write 0xb9c934d thru 0xb9d28a2 (0x9556 bytes) 2024-09-16T13:00:34.667 INFO:teuthology.orchestra.run.smithi042.stdout:4834 punch from 0xa49f722 to 0xa4a204f, (0x292d bytes) 2024-09-16T13:00:34.667 INFO:teuthology.orchestra.run.smithi042.stdout:4835 write 0x9c8d962 thru 0x9c9b115 (0xd7b4 bytes) 2024-09-16T13:00:34.670 INFO:teuthology.orchestra.run.smithi042.stdout:4838 write 0xb8deeb5 thru 0xb8e2374 (0x34c0 bytes) 2024-09-16T13:00:34.671 INFO:teuthology.orchestra.run.smithi042.stdout:4839 write 0x8cd34db thru 0x8cd74b0 (0x3fd6 bytes) 2024-09-16T13:00:34.673 INFO:teuthology.orchestra.run.smithi042.stdout:4840 write 0x89abe00 thru 0x89b90f6 (0xd2f7 bytes) 2024-09-16T13:00:34.676 INFO:teuthology.orchestra.run.smithi042.stdout:4841 read 0xa58df57 thru 0xa59253a (0x45e4 bytes) 2024-09-16T13:00:34.677 INFO:teuthology.orchestra.run.smithi042.stdout:4843 punch from 0x14b4df9 to 0x14bc180, (0x7387 bytes) 2024-09-16T13:00:34.684 INFO:teuthology.orchestra.run.smithi042.stdout:4844 trunc from 0xbe19ee7 to 0x71cb221 2024-09-16T13:00:34.702 INFO:teuthology.orchestra.run.smithi042.stdout:4845 punch from 0x5668b5b to 0x5676b11, (0xdfb6 bytes) 2024-09-16T13:00:34.702 INFO:teuthology.orchestra.run.smithi042.stdout:4847 write 0x5c912f4 thru 0x5c938f9 (0x2606 bytes) 2024-09-16T13:00:34.704 INFO:teuthology.orchestra.run.smithi042.stdout:4849 punch from 0x69857bc to 0x698649c, (0xce0 bytes) 2024-09-16T13:00:34.704 INFO:teuthology.orchestra.run.smithi042.stdout:4851 read 0x2d2d76f thru 0x2d3c6b3 (0xef45 bytes) 2024-09-16T13:00:34.704 INFO:teuthology.orchestra.run.smithi042.stdout:4853 punch from 0x1a03fd9 to 0x1a1382a, (0xf851 bytes) 2024-09-16T13:00:34.719 INFO:teuthology.orchestra.run.smithi042.stdout:4854 write 0xb2815da thru 0xb284622 (0x3049 bytes) 2024-09-16T13:00:34.729 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:34.721+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:34.737 INFO:teuthology.orchestra.run.smithi042.stdout:4855 trunc from 0xb284623 to 0x509558b 2024-09-16T13:00:34.740 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:34.740 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:34.740 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:34.740 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:34.750 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:34.778 INFO:teuthology.orchestra.run.smithi042.stdout:4856 write 0xd066da8 thru 0xd071ab4 (0xad0d bytes) 2024-09-16T13:00:34.782 INFO:teuthology.orchestra.run.smithi042.stdout:4857 read 0x89af2e9 thru 0x89b9e1b (0xab33 bytes) 2024-09-16T13:00:34.783 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:34.783 INFO:teuthology.orchestra.run.smithi042.stdout:4859 read 0x464d6b0 thru 0x4659076 (0xb9c7 bytes) 2024-09-16T13:00:34.795 INFO:teuthology.orchestra.run.smithi042.stdout:4860 trunc from 0xd071ab5 to 0x641c512 2024-09-16T13:00:34.796 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:34.817 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:34.824 INFO:teuthology.orchestra.run.smithi042.stdout:4862 trunc from 0x641c512 to 0x1eec124 2024-09-16T13:00:34.829 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:34.824+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:34.835 INFO:teuthology.orchestra.run.smithi042.stdout:4863 read 0x6bd4bc thru 0x6cad78 (0xd8bd bytes) 2024-09-16T13:00:34.836 INFO:teuthology.orchestra.run.smithi042.stdout:4864 read 0x3cb53c thru 0x3ce77e (0x3243 bytes) 2024-09-16T13:00:34.837 INFO:teuthology.orchestra.run.smithi042.stdout:4865 write 0x110f785 thru 0x11111e3 (0x1a5f bytes) 2024-09-16T13:00:34.839 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:34.857 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:34.857 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:34.859 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:34.861 INFO:teuthology.orchestra.run.smithi042.stdout:4866 write 0xa812655 thru 0xa81b4ed (0x8e99 bytes) 2024-09-16T13:00:34.866 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:34.867 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:34.867 INFO:teuthology.orchestra.run.smithi042.stdout:4869 write 0x951bcc4 thru 0x952aa37 (0xed74 bytes) 2024-09-16T13:00:34.870 INFO:teuthology.orchestra.run.smithi042.stdout:4870 punch from 0x8ee34a8 to 0x8ee5f93, (0x2aeb bytes) 2024-09-16T13:00:34.875 INFO:teuthology.orchestra.run.smithi042.stdout:4871 trunc from 0xa81b4ee to 0x7e929df 2024-09-16T13:00:34.879 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:34.899 INFO:teuthology.orchestra.run.smithi042.stdout:4872 clone 30 order 22 su 65536 sc 4 2024-09-16T13:00:34.904 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:34.925 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:34.926 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:34.927 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:34.927 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:34.930 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:34.925+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:34.956 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:34.956 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:34.965 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:35.031 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:35.024+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:35.083 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:35.083 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:35.084 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:35.132 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:35.124+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:35.168 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2024-09-16T13:00:35.184 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:35.184 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:35.194 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:35.238 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:35.233+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:35.289 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:35.290 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:35.299 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:35.345 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:35.340+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:35.362 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T13:00:35.419 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:35.419 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:35.430 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:35.434 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:35.429+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:35.534 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:35.534 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:35.535 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:35.528+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:35.544 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone29 from 0x7e929df (overlap 0x8b3e44) to 0x459f22 2024-09-16T13:00:35.545 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:35.636 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:35.628+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:35.642 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:35.643 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:35.652 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:35.722 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op apply-layout-settings --pool pool_client.0 2024-09-16T13:00:35.737 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:35.732+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:35.774 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-16T13:00:35.785 INFO:teuthology.orchestra.run.smithi042.stdout:4873 writesame 0x2357193 thru 0x235bf22 data_size 0x1(0x4d90 bytes) 2024-09-16T13:00:35.812 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:35.812 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:35.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:35.837 INFO:teuthology.orchestra.run.smithi042.stdout:4874 write 0xb5c9a5d thru 0xb5cdc43 (0x41e7 bytes) 2024-09-16T13:00:35.838 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:35.832+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:35.845 INFO:teuthology.orchestra.run.smithi042.stdout:4878 read 0x6a659f7 thru 0x6a66560 (0xb6a bytes) 2024-09-16T13:00:35.846 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:35.846 INFO:teuthology.orchestra.run.smithi042.stdout:4880 read 0x66eb51b thru 0x66f6cee (0xb7d4 bytes) 2024-09-16T13:00:35.868 INFO:teuthology.orchestra.run.smithi042.stdout:4882 trunc from 0xb5cdc44 to 0xd9ab0f 2024-09-16T13:00:35.926 INFO:teuthology.orchestra.run.smithi042.stdout:4885 read 0xa47d44 thru 0xa578e5 (0xfba2 bytes) 2024-09-16T13:00:35.938 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:35.938 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:35.938 INFO:teuthology.orchestra.run.smithi042.stdout:4886 trunc from 0xd9ab0f to 0x7fc2ef1 2024-09-16T13:00:35.939 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:35.933+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:35.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:35.954 INFO:teuthology.orchestra.run.smithi042.stdout:4887 read 0x868c51 thru 0x87264d (0x99fd bytes) 2024-09-16T13:00:35.956 INFO:teuthology.orchestra.run.smithi042.stdout:4889 punch from 0x6382878 to 0x638d94b, (0xb0d3 bytes) 2024-09-16T13:00:35.956 INFO:teuthology.orchestra.run.smithi042.stdout:4892 punch from 0x42d8d11 to 0x42d99ee, (0xcdd bytes) 2024-09-16T13:00:35.959 INFO:teuthology.orchestra.run.smithi042.stdout:4893 trunc from 0x7fc2ef1 to 0xa3ff6ef 2024-09-16T13:00:35.978 INFO:teuthology.orchestra.run.smithi042.stdout:4894 trunc from 0xa3ff6ef to 0x6670182 2024-09-16T13:00:35.990 INFO:teuthology.orchestra.run.smithi042.stdout:4896 punch from 0x2ecb6a3 to 0x2ed7fa2, (0xc8ff bytes) 2024-09-16T13:00:35.994 INFO:teuthology.orchestra.run.smithi042.stdout:4897 trunc from 0x6670182 to 0x86b21b2 2024-09-16T13:00:36.007 INFO:teuthology.orchestra.run.smithi042.stdout:4898 writesame 0x4aa13c8 thru 0x4aaa83e data_size 0x67(0x9477 bytes) 2024-09-16T13:00:36.014 INFO:teuthology.orchestra.run.smithi042.stdout:4899 write 0xf03b30 thru 0xf11bf0 (0xe0c1 bytes) 2024-09-16T13:00:36.018 INFO:teuthology.orchestra.run.smithi042.stdout:4901 punch from 0x2b2241d to 0x2b25d06, (0x38e9 bytes) 2024-09-16T13:00:36.018 INFO:teuthology.orchestra.run.smithi042.stdout:4903 read 0x198b76f thru 0x1994fdb (0x986d bytes) 2024-09-16T13:00:36.018 INFO:teuthology.orchestra.run.smithi042.stdout:4904 write 0x8363fa5 thru 0x83683c1 (0x441d bytes) 2024-09-16T13:00:36.025 INFO:teuthology.orchestra.run.smithi042.stdout:4905 write 0x29ddff7 thru 0x29e2001 (0x400b bytes) 2024-09-16T13:00:36.033 INFO:teuthology.orchestra.run.smithi042.stdout:4906 punch from 0x7304d81 to 0x7307dc2, (0x3041 bytes) 2024-09-16T13:00:36.033 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:36.033 INFO:teuthology.orchestra.run.smithi042.stdout:4908 read 0x5a747ff thru 0x5a7b10e (0x6910 bytes) 2024-09-16T13:00:36.040 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:36.033+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:36.053 INFO:teuthology.orchestra.run.smithi042.stdout:4909 writesame 0xca52246 thru 0xca55cab data_size 0x41(0x3a66 bytes) 2024-09-16T13:00:36.054 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:36.054 INFO:teuthology.orchestra.run.smithi042.stdout:4911 read 0x232c343 thru 0x23300b6 (0x3d74 bytes) 2024-09-16T13:00:36.054 INFO:teuthology.orchestra.run.smithi042.stdout:4912 write 0xa1636 thru 0xacf2d (0xb8f8 bytes) 2024-09-16T13:00:36.057 INFO:teuthology.orchestra.run.smithi042.stdout:4913 write 0xb941ff5 thru 0xb94551f (0x352b bytes) 2024-09-16T13:00:36.065 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:36.065 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:36.066 INFO:teuthology.orchestra.run.smithi042.stdout:4915 clone 31 order 18 su 2048 sc 9 2024-09-16T13:00:36.075 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:36.141 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:36.133+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:36.220 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:36.220 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:36.229 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:36.242 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:36.236+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:36.335 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:36.335 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:36.342 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:36.337+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:36.345 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:36.443 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:36.436+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:36.504 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:36.504 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:36.514 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:36.544 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:36.537+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone30 from 0xca55cac (overlap 0xd9ab0f) to 0xc3e6c0 2024-09-16T13:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:36.629 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:36.645 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:36.637+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:36.733 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:36.733 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:36.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:36.745 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:36.737+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:36.844 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:36.845 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:36.846 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:36.837+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:36.854 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:36.857 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-16T13:00:36.875 INFO:teuthology.orchestra.run.smithi042.stdout:4916 punch from 0xa20a321 to 0xa211213, (0x6ef2 bytes) 2024-09-16T13:00:36.876 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2024-09-16T13:00:36.883 INFO:teuthology.orchestra.run.smithi042.stdout:4917 write 0xc7ed091 thru 0xc7f21ce (0x513e bytes) 2024-09-16T13:00:36.904 INFO:teuthology.orchestra.run.smithi042.stdout:4920 writesame 0x2ab22e9 thru 0x2ab7347 data_size 0x5(0x505f bytes) 2024-09-16T13:00:36.929 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:36.946 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:36.941+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:36.950 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:36.951 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:36.964 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:36.998 INFO:teuthology.orchestra.run.smithi042.stdout:4921 trunc from 0xca55cac to 0xf53780 2024-09-16T13:00:37.047 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:37.040+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:37.088 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:37.110 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:37.113 INFO:teuthology.orchestra.run.smithi042.stdout:4923 write 0x4c8a713 thru 0x4c8fbfc (0x54ea bytes) 2024-09-16T13:00:37.125 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:37.125 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:37.129 INFO:teuthology.orchestra.run.smithi042.stdout:4924 read 0x1e4566c thru 0x1e52847 (0xd1dc bytes) 2024-09-16T13:00:37.129 INFO:teuthology.orchestra.run.smithi042.stdout:4928 write 0xf0b971 thru 0xf18731 (0xcdc1 bytes) 2024-09-16T13:00:37.134 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:37.135 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:37.148 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:37.140+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:37.151 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:37.158 INFO:teuthology.orchestra.run.smithi042.stdout:4929 write 0xe6ec194 thru 0xe6f108d (0x4efa bytes) 2024-09-16T13:00:37.172 INFO:teuthology.orchestra.run.smithi042.stdout:4930 read 0x2b4c91a thru 0x2b4cd11 (0x3f8 bytes) 2024-09-16T13:00:37.172 INFO:teuthology.orchestra.run.smithi042.stdout:4931 read 0x6491784 thru 0x6498578 (0x6df5 bytes) 2024-09-16T13:00:37.172 INFO:teuthology.orchestra.run.smithi042.stdout:4932 punch from 0x6e4a723 to 0x6e4dfc9, (0x38a6 bytes) 2024-09-16T13:00:37.172 INFO:teuthology.orchestra.run.smithi042.stdout:4934 writesame 0x2b6c94f thru 0x2b7c163 data_size 0x1(0xf815 bytes) 2024-09-16T13:00:37.186 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:37.191 INFO:teuthology.orchestra.run.smithi042.stdout:4935 read 0x630143b thru 0x6304ea0 (0x3a66 bytes) 2024-09-16T13:00:37.191 INFO:teuthology.orchestra.run.smithi042.stdout:4938 read 0x90bb58e thru 0x90c775e (0xc1d1 bytes) 2024-09-16T13:00:37.191 INFO:teuthology.orchestra.run.smithi042.stdout:4939 write 0xcc14f17 thru 0xcc1a7b4 (0x589e bytes) 2024-09-16T13:00:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:4941 read 0x19ec5b0 thru 0x19f5e65 (0x98b6 bytes) 2024-09-16T13:00:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:37.209 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:37.215 INFO:teuthology.orchestra.run.smithi042.stdout:4945 writesame 0xebb46e3 thru 0xebc457a data_size 0x4(0xfe98 bytes) 2024-09-16T13:00:37.215 INFO:teuthology.orchestra.run.smithi042.stdout:4948 read 0x529a532 thru 0x52a287f (0x834e bytes) 2024-09-16T13:00:37.215 INFO:teuthology.orchestra.run.smithi042.stdout:4949 read 0x88573c0 thru 0x8860e7e (0x9abf bytes) 2024-09-16T13:00:37.215 INFO:teuthology.orchestra.run.smithi042.stdout:4950 read 0x3d32828 thru 0x3d40cda (0xe4b3 bytes) 2024-09-16T13:00:37.216 INFO:teuthology.orchestra.run.smithi042.stdout:4951 punch from 0x8be9912 to 0x8bf011a, (0x6808 bytes) 2024-09-16T13:00:37.216 INFO:teuthology.orchestra.run.smithi042.stdout:4954 write 0x6a924bb thru 0x6a9b83e (0x9384 bytes) 2024-09-16T13:00:37.234 INFO:teuthology.orchestra.run.smithi042.stdout:4955 read 0x6676ecf thru 0x667fd8c (0x8ebe bytes) 2024-09-16T13:00:37.237 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:37.237 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:37.237 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:37.237 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:37.238 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:37.238 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:37.238 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:37.238 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:37.238 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:37.239 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:37.239 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:37.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:37.239 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:37.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.239 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:8419 4.7 4:e1fbf1ee:::rbd_data.11b55eb65e78.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T13:00:37.239 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:37.239 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.831390794999997, 2024-09-16T13:00:37.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040293592000000003, 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 8419 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:37.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402892+0000", 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402894+0000", 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3300000000000001e-06 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402907+0000", 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402940+0000", 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3436000000000003e-05 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.402953+0000", 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2840000000000001e-05 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403104+0000", 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150712 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.403202+0000", 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8403e-05 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.411015+0000", 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078133079999999997 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443018+0000", 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032002348999999999 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443075+0000", 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7703e-05 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443141+0000", 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5705999999999997e-05 2024-09-16T13:00:37.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:36.443185+0000", 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4152e-05 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.268158437, 2024-09-16T13:00:37.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039945895000000002, 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48384 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966124+0000", 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966126+0000", 2024-09-16T13:00:37.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1110000000000002e-06 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966134+0000", 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9740000000000002e-06 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966139+0000", 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4270000000000001e-06 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.005829+0000", 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039690007999999999 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.005928+0000", 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9184000000000004e-05 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006070+0000", 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014171100000000001 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.249 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.268138806, 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040120731, 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.250 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48385 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966144+0000", 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.63e-07 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966146+0000", 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.018e-06 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966154+0000", 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9999999999999996e-06 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966169+0000", 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5395000000000001e-05 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006118+0000", 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039948637000000002 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006181+0000", 2024-09-16T13:00:37.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3001000000000002e-05 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006264+0000", 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3217e-05 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.268107683, 2024-09-16T13:00:37.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040378561, 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48386 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966175+0000", 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5199999999999996e-07 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966178+0000", 2024-09-16T13:00:37.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.373e-06 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966186+0000", 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3259999999999996e-06 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966202+0000", 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6333000000000001e-05 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006311+0000", 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040108447999999998 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006411+0000", 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100141 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006553+0000", 2024-09-16T13:00:37.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014218799999999999 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.268074152000001, 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040537542000000003, 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48387 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966208+0000", 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966209+0000", 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4300000000000003e-07 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966211+0000", 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.418e-06 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966219+0000", 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.627e-06 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966223+0000", 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3019999999999996e-06 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006601+0000", 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040377953000000001 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006665+0000", 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3592999999999999e-05 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006746+0000", 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1106000000000006e-05 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.263 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.268054553999999, 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040793276000000003, 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48388 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966228+0000", 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5700000000000002e-07 2024-09-16T13:00:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966230+0000", 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0499999999999999e-06 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966246+0000", 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5543e-05 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966250+0000", 2024-09-16T13:00:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0840000000000002e-06 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006794+0000", 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040543691999999999 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.006901+0000", 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107754 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007021+0000", 2024-09-16T13:00:37.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119596 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.268027153999999, 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040958857000000001, 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.268 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48389 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966255+0000", 2024-09-16T13:00:37.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966256+0000", 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8100000000000003e-07 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966258+0000", 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3829999999999999e-06 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966268+0000", 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8379999999999997e-06 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966272+0000", 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9940000000000002e-06 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007069+0000", 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040796897999999998 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007132+0000", 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3084999999999997e-05 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007214+0000", 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2177999999999998e-05 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.272 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.268004676, 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041219483000000001, 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48390 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966278+0000", 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4700000000000002e-07 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966280+0000", 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3e-06 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966286+0000", 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.57e-06 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966288+0000", 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4439999999999998e-06 2024-09-16T13:00:37.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007262+0000", 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040973242 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007351+0000", 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9339999999999995e-05 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007497+0000", 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014614000000000001 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267983012999998, 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041394590000000002, 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.277 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48391 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966299+0000", 2024-09-16T13:00:37.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966300+0000", 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966301+0000", 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5510000000000001e-06 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966305+0000", 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1849999999999997e-06 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966310+0000", 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.211e-06 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007546+0000", 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041236341000000003 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007610+0000", 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3534999999999994e-05 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007694+0000", 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4407999999999996e-05 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.281 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48392 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 794624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267970359, 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041645095, 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48392 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966312+0000", 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3599999999999999e-07 2024-09-16T13:00:37.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966314+0000", 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7689999999999999e-06 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966322+0000", 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9340000000000006e-06 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966325+0000", 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5579999999999999e-06 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007742+0000", 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041417127999999998 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007840+0000", 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8199e-05 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.007957+0000", 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011717099999999999 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48393 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267955163, 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041849738999999997, 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.286 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48393 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966327+0000", 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966328+0000", 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966340+0000", 2024-09-16T13:00:37.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2604e-05 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966344+0000", 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1989999999999999e-06 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966348+0000", 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4039999999999999e-06 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008030+0000", 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041682036999999998 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008093+0000", 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3511999999999998e-05 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008177+0000", 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3561999999999997e-05 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.290 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48394 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267931703999999, 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042095443000000003, 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.291 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48394 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966351+0000", 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966352+0000", 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.406e-06 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966356+0000", 2024-09-16T13:00:37.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966362+0000", 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2299999999999996e-06 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008232+0000", 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041869361000000001 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008320+0000", 2024-09-16T13:00:37.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8492000000000006e-05 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008446+0000", 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012591800000000001 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48395 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:37.295 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267916162999999, 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042265693999999999, 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48395 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:37.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966366+0000", 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966367+0000", 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966368+0000", 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9420000000000002e-06 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966372+0000", 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9049999999999999e-06 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966375+0000", 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4329999999999998e-06 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008483+0000", 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042108244000000003 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008546+0000", 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2973999999999997e-05 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008632+0000", 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.585e-05 2024-09-16T13:00:37.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48396 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267905076000002, 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042505605000000002, 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.300 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48396 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966377+0000", 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966378+0000", 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966379+0000", 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.147e-06 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966382+0000", 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966385+0000", 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.43e-06 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008677+0000", 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042292191 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008765+0000", 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8027000000000003e-05 2024-09-16T13:00:37.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008883+0000", 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117817 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48397 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:37.304 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267895169999999, 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042627777999999998, 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48397 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:37.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966387+0000", 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966388+0000", 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966389+0000", 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.144e-06 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966392+0000", 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5939999999999998e-06 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966398+0000", 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3959999999999997e-06 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008904+0000", 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042505730999999998 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.008953+0000", 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9786000000000001e-05 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009015+0000", 2024-09-16T13:00:37.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1580000000000003e-05 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48398 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267881828, 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043115957000000003, 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48398 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966401+0000", 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2500000000000001e-07 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966402+0000", 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3120000000000001e-06 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966406+0000", 2024-09-16T13:00:37.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.653e-06 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966408+0000", 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.322e-06 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009095+0000", 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042687260999999997 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009276+0000", 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000180345 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009516+0000", 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024073899999999999 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48399 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.313 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267871095, 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043478389999999999, 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48399 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966411+0000", 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966412+0000", 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.03e-07 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966413+0000", 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2410000000000001e-06 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966416+0000", 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6330000000000002e-06 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966422+0000", 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2379999999999997e-06 2024-09-16T13:00:37.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009539+0000", 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043116865999999997 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009588+0000", 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9731999999999998e-05 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009890+0000", 2024-09-16T13:00:37.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030137700000000003 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48400 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267857365000001, 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043788502, 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.318 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48400 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966425+0000", 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3999999999999997e-07 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966427+0000", 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.765e-06 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966434+0000", 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6989999999999997e-06 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966437+0000", 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1109999999999999e-06 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.009969+0000", 2024-09-16T13:00:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043532556 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010101+0000", 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0001317 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010213+0000", 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011233100000000001 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48401 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 1015808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267842753, 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043965861000000002, 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48401 2024-09-16T13:00:37.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966440+0000", 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966442+0000", 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8300000000000001e-06 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966455+0000", 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2927e-05 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966459+0000", 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3850000000000004e-06 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010261+0000", 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043801305999999998 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010335+0000", 2024-09-16T13:00:37.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4160999999999997e-05 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010405+0000", 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0716000000000005e-05 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:48402 4.8 4:147b8303:::rbd_data.11b5b3dc7343.0000000000000017:19 [read 1040384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:37.327 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.267818255000002, 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044179172000000003, 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48402 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.966464+0000", 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969465+0000", 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003000689 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969469+0000", 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5739999999999997e-06 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969483+0000", 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3692e-05 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:10.969488+0000", 2024-09-16T13:00:37.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6450000000000004e-06 2024-09-16T13:00:37.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010444+0000", 2024-09-16T13:00:37.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040956030999999997 2024-09-16T13:00:37.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010542+0000", 2024-09-16T13:00:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8486000000000005e-05 2024-09-16T13:00:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:11.010643+0000", 2024-09-16T13:00:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101055 2024-09-16T13:00:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.333 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.333 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:37.334 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:37.335 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:37.335 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:37.335 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:37.325+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:37.336 INFO:teuthology.orchestra.run.smithi042.stdout:4956 trunc from 0xebc457b to 0x7e13588 2024-09-16T13:00:37.336 INFO:teuthology.orchestra.run.smithi042.stdout:4957 write 0xe2f88f8 thru 0xe300fbb (0x86c4 bytes) 2024-09-16T13:00:37.336 INFO:teuthology.orchestra.run.smithi042.stdout:4958 write 0xa4fad06 thru 0xa4fd04b (0x2346 bytes) 2024-09-16T13:00:37.336 INFO:teuthology.orchestra.run.smithi042.stdout:4960 write 0x3e08b22 thru 0x3e1666d (0xdb4c bytes) 2024-09-16T13:00:37.336 INFO:teuthology.orchestra.run.smithi042.stdout:4962 punch from 0x7375123 to 0x73776e7, (0x25c4 bytes) 2024-09-16T13:00:37.336 INFO:teuthology.orchestra.run.smithi042.stdout:4963 writesame 0x41ac8c4 thru 0x41b49ff data_size 0x12(0x813c bytes) 2024-09-16T13:00:37.348 INFO:teuthology.orchestra.run.smithi042.stdout:4967 write 0x6421100 thru 0x642ee0e (0xdd0f bytes) 2024-09-16T13:00:37.361 INFO:teuthology.orchestra.run.smithi042.stdout:4968 flatten 2024-09-16T13:00:37.376 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T13:00:37.435 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:37.428+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:37.486 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:37.486 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:37.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.486 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233463 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.486 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.455428+0000", 2024-09-16T13:00:37.486 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.010268179000000001, 2024-09-16T13:00:37.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010296036999999999, 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233463 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455428+0000", 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455428+0000", 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455429+0000", 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7e-07 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455431+0000", 2024-09-16T13:00:37.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.854e-06 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455436+0000", 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455439+0000", 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2389999999999998e-06 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.489 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233496 4.18 4.d4aa6578 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.455984+0000", 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0097127610000000003, 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097928730000000005, 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233496 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455984+0000", 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455984+0000", 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455984+0000", 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456102+0000", 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011795599999999999 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456109+0000", 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2869999999999999e-06 2024-09-16T13:00:37.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456113+0000", 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9029999999999997e-06 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233530 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456672+0000", 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0090246890000000007, 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009140469, 2024-09-16T13:00:37.493 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233530 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456672+0000", 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456672+0000", 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456672+0000", 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3300000000000002e-07 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456675+0000", 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1449999999999998e-06 2024-09-16T13:00:37.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456680+0000", 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5620000000000003e-06 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456684+0000", 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233626 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457294+0000", 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0084027589999999992, 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085553450000000007, 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233626 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.497 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457294+0000", 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457294+0000", 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457294+0000", 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:37.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457296+0000", 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9439999999999999e-06 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457302+0000", 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.319e-06 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457306+0000", 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6160000000000002e-06 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233661 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457861+0000", 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0078357889999999993, 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008040841, 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233661 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457861+0000", 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457861+0000", 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457861+0000", 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9900000000000001e-07 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457863+0000", 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0509999999999999e-06 2024-09-16T13:00:37.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457869+0000", 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457873+0000", 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5930000000000002e-06 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233757 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458397+0000", 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.007299926, 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075766089999999998, 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233757 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458397+0000", 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458397+0000", 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458397+0000", 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.89e-07 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458399+0000", 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8819999999999999e-06 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458404+0000", 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1080000000000001e-06 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458407+0000", 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2880000000000001e-06 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233789 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.461272+0000", 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0044244360000000003, 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004771038, 2024-09-16T13:00:37.507 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233789 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461272+0000", 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461272+0000", 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461475+0000", 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000203174 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461479+0000", 2024-09-16T13:00:37.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2710000000000001e-06 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461487+0000", 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461510+0000", 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.321e-05 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233464 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.455443+0000", 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.010253632, 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010672676000000001, 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233464 2024-09-16T13:00:37.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455443+0000", 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455443+0000", 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455443+0000", 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455451+0000", 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3710000000000002e-06 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455457+0000", 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9370000000000001e-06 2024-09-16T13:00:37.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455460+0000", 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5209999999999998e-06 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233499 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456119+0000", 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0095775070000000007, 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010066245, 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.514 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233499 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456119+0000", 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456119+0000", 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456120+0000", 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7599999999999997e-07 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456122+0000", 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0770000000000002e-06 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456127+0000", 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2399999999999998e-06 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456130+0000", 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3639999999999999e-06 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.517 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233595 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456687+0000", 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0090091070000000006, 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095578639999999992, 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233595 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456687+0000", 2024-09-16T13:00:37.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456687+0000", 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456688+0000", 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456690+0000", 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.835e-06 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456695+0000", 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7919999999999998e-06 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456699+0000", 2024-09-16T13:00:37.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233630 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457310+0000", 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.008386981, 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089709270000000001, 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.521 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.522 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.522 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233630 2024-09-16T13:00:37.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.522 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457310+0000", 2024-09-16T13:00:37.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457310+0000", 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457310+0000", 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7300000000000002e-07 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457312+0000", 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.751e-06 2024-09-16T13:00:37.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457318+0000", 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9710000000000001e-06 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457321+0000", 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7290000000000002e-06 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233726 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457876+0000", 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0078203879999999993, 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084389789999999992, 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233726 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.525 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457876+0000", 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457876+0000", 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457877+0000", 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6800000000000001e-07 2024-09-16T13:00:37.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457879+0000", 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9319999999999999e-06 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457884+0000", 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3700000000000003e-06 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457887+0000", 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.23e-06 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233758 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458411+0000", 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0072855100000000002, 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079382770000000005, 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.528 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233758 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458411+0000", 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458411+0000", 2024-09-16T13:00:37.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458411+0000", 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9000000000000002e-07 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458413+0000", 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458420+0000", 2024-09-16T13:00:37.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1759999999999998e-06 2024-09-16T13:00:37.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458425+0000", 2024-09-16T13:00:37.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.977e-06 2024-09-16T13:00:37.531 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.531 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233854 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.461516+0000", 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.004180614, 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048674959999999998, 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.532 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233854 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461516+0000", 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461516+0000", 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461517+0000", 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2099999999999996e-07 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461519+0000", 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.571e-06 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461525+0000", 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8610000000000003e-06 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461531+0000", 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7219999999999996e-06 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233465 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.535 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.455477+0000", 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.010219054, 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010942443, 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233465 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455477+0000", 2024-09-16T13:00:37.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455477+0000", 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455478+0000", 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.37e-07 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455480+0000", 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1100000000000001e-06 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455486+0000", 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0279999999999997e-06 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455491+0000", 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5419999999999999e-06 2024-09-16T13:00:37.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233500 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456136+0000", 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0095603749999999994, 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010341196, 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.539 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233500 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456136+0000", 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456136+0000", 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456137+0000", 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.08e-07 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456139+0000", 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0099999999999998e-06 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456146+0000", 2024-09-16T13:00:37.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0330000000000003e-06 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456149+0000", 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7060000000000002e-06 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233596 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456703+0000", 2024-09-16T13:00:37.542 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0089932359999999999, 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098409259999999998, 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233596 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456703+0000", 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456703+0000", 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456704+0000", 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456706+0000", 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0770000000000002e-06 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456711+0000", 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5010000000000004e-06 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456715+0000", 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6059999999999999e-06 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233631 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457326+0000", 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0083704869999999994, 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092891789999999998, 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233631 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.546 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457326+0000", 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457326+0000", 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457326+0000", 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2300000000000002e-07 2024-09-16T13:00:37.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457328+0000", 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9870000000000002e-06 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457334+0000", 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2599999999999996e-06 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457337+0000", 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4599999999999999e-06 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233727 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457891+0000", 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0078055920000000001, 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087984010000000008, 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.549 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233727 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457891+0000", 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457891+0000", 2024-09-16T13:00:37.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457891+0000", 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457897+0000", 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0580000000000003e-06 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457903+0000", 2024-09-16T13:00:37.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8850000000000004e-06 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457907+0000", 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5590000000000001e-06 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233759 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458429+0000", 2024-09-16T13:00:37.552 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0072678999999999999, 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008328986, 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233759 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458429+0000", 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458429+0000", 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458429+0000", 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458431+0000", 2024-09-16T13:00:37.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0250000000000001e-06 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458437+0000", 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5040000000000002e-06 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458440+0000", 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6279999999999998e-06 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233855 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.461535+0000", 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0041617579999999998, 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00528905, 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233855 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.556 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461535+0000", 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461535+0000", 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461535+0000", 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461537+0000", 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.097e-06 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461543+0000", 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7939999999999999e-06 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461547+0000", 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0570000000000003e-06 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233466 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 663552~2048 out=2048b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.455495+0000", 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.010201963, 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011400156, 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.559 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233466 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455495+0000", 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455495+0000", 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455495+0000", 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5300000000000004e-07 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455497+0000", 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7749999999999999e-06 2024-09-16T13:00:37.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455503+0000", 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455506+0000", 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8580000000000002e-06 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.466744+0000", 2024-09-16T13:00:37.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011237087 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.466818+0000", 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4130999999999999e-05 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233501 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456153+0000", 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0095432999999999994, 2024-09-16T13:00:37.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010818715, 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233501 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456153+0000", 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456153+0000", 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456154+0000", 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.01e-07 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456156+0000", 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1119999999999999e-06 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456161+0000", 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5639999999999996e-06 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456165+0000", 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.315e-06 2024-09-16T13:00:37.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233597 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456718+0000", 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0089781059999999996, 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010314169, 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.567 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233597 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456718+0000", 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456718+0000", 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456719+0000", 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4600000000000005e-07 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456721+0000", 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8670000000000001e-06 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456726+0000", 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6359999999999999e-06 2024-09-16T13:00:37.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456730+0000", 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3170000000000002e-06 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233632 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457341+0000", 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.008355866, 2024-09-16T13:00:37.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097561439999999996, 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233632 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457341+0000", 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457341+0000", 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457341+0000", 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457343+0000", 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.844e-06 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457350+0000", 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9800000000000001e-06 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457353+0000", 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5279999999999999e-06 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.573 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233728 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457911+0000", 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0077857339999999999, 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092468350000000001, 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233728 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.574 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457911+0000", 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457911+0000", 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457911+0000", 2024-09-16T13:00:37.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7599999999999998e-07 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457913+0000", 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8339999999999999e-06 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457918+0000", 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3449999999999996e-06 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457922+0000", 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4029999999999999e-06 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233760 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458444+0000", 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0072529760000000004, 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087744109999999993, 2024-09-16T13:00:37.577 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233760 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458444+0000", 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458444+0000", 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458444+0000", 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8700000000000001e-07 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458446+0000", 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8759999999999999e-06 2024-09-16T13:00:37.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458451+0000", 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3979999999999998e-06 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458455+0000", 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4989999999999999e-06 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233856 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.461551+0000", 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0041456669999999996, 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057287509999999998, 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233856 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.581 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461551+0000", 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461551+0000", 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461655+0000", 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103743 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461658+0000", 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4290000000000001e-06 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461665+0000", 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0690000000000004e-06 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461670+0000", 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6940000000000003e-06 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233502 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456168+0000", 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0095283550000000005, 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011178347, 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.584 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233502 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456168+0000", 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456168+0000", 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456169+0000", 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456170+0000", 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.886e-06 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456176+0000", 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3770000000000004e-06 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456179+0000", 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3919999999999999e-06 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233598 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456733+0000", 2024-09-16T13:00:37.587 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0089630500000000002, 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010679144999999999, 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233598 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456733+0000", 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456733+0000", 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456734+0000", 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.58e-07 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456736+0000", 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8670000000000001e-06 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456741+0000", 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4129999999999997e-06 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456745+0000", 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4510000000000001e-06 2024-09-16T13:00:37.590 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233633 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457357+0000", 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0083394390000000006, 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010119215000000001, 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.591 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233633 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457357+0000", 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457357+0000", 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457357+0000", 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7099999999999997e-07 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457359+0000", 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.962e-06 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457365+0000", 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6339999999999998e-06 2024-09-16T13:00:37.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457369+0000", 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5729999999999999e-06 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233729 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457925+0000", 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0077713610000000001, 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095968500000000005, 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.594 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233729 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457925+0000", 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457925+0000", 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457926+0000", 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4799999999999998e-07 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457928+0000", 2024-09-16T13:00:37.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1660000000000001e-06 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457933+0000", 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4890000000000003e-06 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457937+0000", 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3500000000000001e-06 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233761 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458458+0000", 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.007238459, 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091092959999999994, 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233761 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.598 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458458+0000", 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458458+0000", 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458458+0000", 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6199999999999999e-07 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458460+0000", 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7630000000000001e-06 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458466+0000", 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3580000000000002e-06 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458469+0000", 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6959999999999998e-06 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233857 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.461675+0000", 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0040218780000000004, 2024-09-16T13:00:37.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059615689999999999, 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233857 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461675+0000", 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461675+0000", 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465481+0000", 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038065500000000001 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465486+0000", 2024-09-16T13:00:37.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0119999999999996e-06 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465500+0000", 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4215e-05 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465507+0000", 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8719999999999996e-06 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.604 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233503 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456182+0000", 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0095140629999999997, 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011529271000000001, 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233503 2024-09-16T13:00:37.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456182+0000", 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456182+0000", 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456183+0000", 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7099999999999997e-07 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456185+0000", 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.945e-06 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456190+0000", 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4410000000000001e-06 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456194+0000", 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5319999999999998e-06 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233599 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456748+0000", 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.008948147, 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011044459, 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.608 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233599 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456748+0000", 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456748+0000", 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456749+0000", 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456754+0000", 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5629999999999999e-06 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456761+0000", 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9360000000000002e-06 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456765+0000", 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.433e-06 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.611 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233634 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457392+0000", 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0083043170000000003, 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010471261000000001, 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233634 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457392+0000", 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457392+0000", 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457393+0000", 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5499999999999998e-07 2024-09-16T13:00:37.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457395+0000", 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.508e-06 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457403+0000", 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3329999999999999e-06 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457407+0000", 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8419999999999998e-06 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233730 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457940+0000", 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0077561890000000001, 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010001988, 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233730 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457940+0000", 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457940+0000", 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457941+0000", 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.99e-07 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457943+0000", 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0609999999999998e-06 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457950+0000", 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9689999999999997e-06 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457953+0000", 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3469999999999999e-06 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233762 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458473+0000", 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0072236849999999997, 2024-09-16T13:00:37.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095359989999999999, 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233762 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458473+0000", 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458473+0000", 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458473+0000", 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.58e-07 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458475+0000", 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.88e-06 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458480+0000", 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3079999999999998e-06 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458484+0000", 2024-09-16T13:00:37.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3979999999999999e-06 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233858 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465524+0000", 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.00017271699999999999, 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002550783, 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.622 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233858 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465524+0000", 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465524+0000", 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465525+0000", 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.61e-07 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465527+0000", 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.875e-06 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465534+0000", 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6340000000000003e-06 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465538+0000", 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3680000000000004e-06 2024-09-16T13:00:37.625 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233504 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456217+0000", 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0094799110000000006, 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011894919, 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233504 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456217+0000", 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456217+0000", 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456217+0000", 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1600000000000001e-07 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456220+0000", 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2369999999999999e-06 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456226+0000", 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7610000000000002e-06 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456231+0000", 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3669999999999999e-06 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.629 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233600 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456769+0000", 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0089279949999999993, 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011376964, 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233600 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456769+0000", 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456769+0000", 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456769+0000", 2024-09-16T13:00:37.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.01e-07 2024-09-16T13:00:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456771+0000", 2024-09-16T13:00:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.001e-06 2024-09-16T13:00:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456776+0000", 2024-09-16T13:00:37.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2599999999999996e-06 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456780+0000", 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7129999999999999e-06 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233635 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457410+0000", 2024-09-16T13:00:37.633 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0082862349999999994, 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010768402999999999, 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233635 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457410+0000", 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457410+0000", 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457411+0000", 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457413+0000", 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9410000000000001e-06 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457420+0000", 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.542e-06 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457424+0000", 2024-09-16T13:00:37.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2069999999999997e-06 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233731 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457957+0000", 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0077396139999999997, 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010255113, 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233731 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457957+0000", 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457957+0000", 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457958+0000", 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7299999999999995e-07 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457959+0000", 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457965+0000", 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.75e-06 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457969+0000", 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.388e-06 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.640 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233763 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458487+0000", 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0072093809999999999, 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097579099999999998, 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233763 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458487+0000", 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458487+0000", 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458487+0000", 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:37.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458494+0000", 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2009999999999996e-06 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458502+0000", 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1389999999999995e-06 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458505+0000", 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5360000000000001e-06 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233859 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465543+0000", 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.000153979, 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027354879999999999, 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.644 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233859 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465543+0000", 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465543+0000", 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465543+0000", 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9800000000000003e-07 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465545+0000", 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0609999999999998e-06 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465551+0000", 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8100000000000003e-06 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465555+0000", 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.659e-06 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.647 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233470 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.455557+0000", 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.010139267, 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012755513, 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233470 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455557+0000", 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455557+0000", 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455558+0000", 2024-09-16T13:00:37.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6800000000000001e-07 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455559+0000", 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7880000000000001e-06 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455565+0000", 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3850000000000001e-06 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455568+0000", 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.371e-06 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233505 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456236+0000", 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0094607709999999998, 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012112692, 2024-09-16T13:00:37.651 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233505 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456236+0000", 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456236+0000", 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456236+0000", 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7599999999999998e-07 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456238+0000", 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9530000000000002e-06 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456244+0000", 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5260000000000001e-06 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456247+0000", 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3680000000000002e-06 2024-09-16T13:00:37.654 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233601 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456784+0000", 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0089128660000000002, 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011619539999999999, 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.655 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233601 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456784+0000", 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456784+0000", 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456784+0000", 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3e-07 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456793+0000", 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6859999999999995e-06 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456799+0000", 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2179999999999996e-06 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456803+0000", 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.743e-06 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233636 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457429+0000", 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0082680140000000006, 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011042346, 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233636 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457429+0000", 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457429+0000", 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457429+0000", 2024-09-16T13:00:37.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.89e-07 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457431+0000", 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.846e-06 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457436+0000", 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5130000000000004e-06 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457440+0000", 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.269e-06 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233732 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457972+0000", 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0077243219999999996, 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010566618999999999, 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233732 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457972+0000", 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457972+0000", 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457973+0000", 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.53e-07 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457974+0000", 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.77e-06 2024-09-16T13:00:37.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457980+0000", 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3329999999999996e-06 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457983+0000", 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4809999999999998e-06 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233764 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458509+0000", 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0071870390000000001, 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010098664, 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233764 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458509+0000", 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458509+0000", 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458807+0000", 2024-09-16T13:00:37.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029775400000000001 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458811+0000", 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5020000000000001e-06 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458819+0000", 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5129999999999996e-06 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458824+0000", 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.673e-06 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233860 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465560+0000", 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.00013687700000000001, 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031160850000000002, 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233860 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465560+0000", 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465560+0000", 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465560+0000", 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.65e-07 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465562+0000", 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0509999999999999e-06 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465568+0000", 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.451e-06 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465572+0000", 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8069999999999999e-06 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233471 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.455572+0000", 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.010124917000000001, 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013169973, 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233471 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455572+0000", 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455572+0000", 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455572+0000", 2024-09-16T13:00:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455574+0000", 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455579+0000", 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4569999999999996e-06 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455583+0000", 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4199999999999999e-06 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233506 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456250+0000", 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0094460659999999995, 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012545992000000001, 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233506 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456250+0000", 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456250+0000", 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456251+0000", 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456253+0000", 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8059999999999999e-06 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456258+0000", 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3170000000000001e-06 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456261+0000", 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3469999999999999e-06 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233602 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456807+0000", 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0088898350000000004, 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012056730999999999, 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233602 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456807+0000", 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456807+0000", 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456807+0000", 2024-09-16T13:00:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3499999999999996e-07 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456809+0000", 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9190000000000002e-06 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456815+0000", 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6879999999999996e-06 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456818+0000", 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2880000000000001e-06 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233637 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457444+0000", 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0082521369999999997, 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01148618, 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233637 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457444+0000", 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457444+0000", 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457445+0000", 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9099999999999999e-07 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457447+0000", 2024-09-16T13:00:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9929999999999998e-06 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457452+0000", 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7219999999999996e-06 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457462+0000", 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8539999999999992e-06 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233733 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458012+0000", 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0076848120000000001, 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010985478999999999, 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233733 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458012+0000", 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458012+0000", 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458012+0000", 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8999999999999996e-07 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458015+0000", 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2520000000000002e-06 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458021+0000", 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3929999999999996e-06 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458025+0000", 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2880000000000003e-06 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233765 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458831+0000", 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0068654919999999999, 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010234437000000001, 2024-09-16T13:00:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233765 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458831+0000", 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458831+0000", 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458832+0000", 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8500000000000002e-07 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458834+0000", 2024-09-16T13:00:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.221e-06 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458840+0000", 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7790000000000001e-06 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458843+0000", 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8310000000000003e-06 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233861 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465576+0000", 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.000120184, 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035407339999999998, 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233861 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465576+0000", 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465576+0000", 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465577+0000", 2024-09-16T13:00:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0400000000000002e-07 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465579+0000", 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0679999999999999e-06 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465584+0000", 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5969999999999999e-06 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465588+0000", 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4649999999999999e-06 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233472 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 761856~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.455586+0000", 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.010110256, 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013578708, 2024-09-16T13:00:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233472 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455586+0000", 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455586+0000", 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455587+0000", 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455589+0000", 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.075e-06 2024-09-16T13:00:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455594+0000", 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3979999999999998e-06 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.455597+0000", 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.157e-06 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469111+0000", 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013514145999999999 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469165+0000", 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3727999999999998e-05 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233507 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456265+0000", 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.009431544, 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012946229, 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233507 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456265+0000", 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456265+0000", 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456265+0000", 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.58e-07 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456267+0000", 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6810000000000001e-06 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456272+0000", 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3489999999999999e-06 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456276+0000", 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6610000000000002e-06 2024-09-16T13:00:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233603 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456824+0000", 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0088725660000000001, 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012429766, 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233603 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456824+0000", 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456824+0000", 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456824+0000", 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.96e-07 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456826+0000", 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.917e-06 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456833+0000", 2024-09-16T13:00:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9750000000000001e-06 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456837+0000", 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3569999999999998e-06 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233638 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457467+0000", 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0082299020000000007, 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011854480000000001, 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233638 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457467+0000", 2024-09-16T13:00:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457467+0000", 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457467+0000", 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5699999999999998e-07 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457475+0000", 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2369999999999992e-06 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457481+0000", 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8459999999999996e-06 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457486+0000", 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2639999999999999e-06 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233734 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458030+0000", 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0076668760000000004, 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011361102, 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233734 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458030+0000", 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458030+0000", 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458030+0000", 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458032+0000", 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.007e-06 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458038+0000", 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4870000000000002e-06 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458041+0000", 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4460000000000001e-06 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233766 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458847+0000", 2024-09-16T13:00:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0068491510000000004, 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01062185, 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233766 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458847+0000", 2024-09-16T13:00:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458847+0000", 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458848+0000", 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458850+0000", 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2680000000000001e-06 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458856+0000", 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.507e-06 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458859+0000", 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233862 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465591+0000", 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.000105152, 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039455790000000003, 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233862 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465591+0000", 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465591+0000", 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465592+0000", 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2599999999999998e-07 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465594+0000", 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.849e-06 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465599+0000", 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7520000000000002e-06 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465603+0000", 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5810000000000001e-06 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4533.0:233887 4.1f e94/31 ondisk, result = 0)", 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.468966+0000", 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9967308, 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026184199999999998, 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.468966+0000", 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.468966+0000", 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.468967+0000", 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1100000000000004e-07 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.468971+0000", 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5090000000000004e-06 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.468982+0000", 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0392e-05 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.468987+0000", 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6210000000000001e-06 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469066+0000", 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8850999999999994e-05 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469083+0000", 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6951000000000001e-05 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469228+0000", 2024-09-16T13:00:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014460699999999999 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233508 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456280+0000", 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0094167529999999999, 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013411001000000001, 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233508 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456280+0000", 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456280+0000", 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456280+0000", 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0499999999999999e-07 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456282+0000", 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0810000000000001e-06 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456289+0000", 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0940000000000002e-06 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456293+0000", 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233604 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456840+0000", 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0088562459999999999, 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012937582, 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233604 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456840+0000", 2024-09-16T13:00:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456840+0000", 2024-09-16T13:00:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456841+0000", 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5200000000000002e-07 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456843+0000", 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8810000000000001e-06 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456848+0000", 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8930000000000002e-06 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456852+0000", 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4939999999999999e-06 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233639 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457491+0000", 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0082059860000000002, 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012346106000000001, 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233639 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457491+0000", 2024-09-16T13:00:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457491+0000", 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457491+0000", 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6899999999999997e-07 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457493+0000", 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0499999999999999e-06 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457499+0000", 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5419999999999996e-06 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457502+0000", 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5209999999999998e-06 2024-09-16T13:00:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233735 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458045+0000", 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0076512480000000003, 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011826011000000001, 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233735 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458045+0000", 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458045+0000", 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458046+0000", 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2800000000000003e-07 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458047+0000", 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.897e-06 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458055+0000", 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4159999999999998e-06 2024-09-16T13:00:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458060+0000", 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.899e-06 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233767 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458862+0000", 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0068340429999999997, 2024-09-16T13:00:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0110416, 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233767 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458862+0000", 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458862+0000", 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458863+0000", 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458865+0000", 2024-09-16T13:00:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458871+0000", 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5300000000000002e-06 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458875+0000", 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6040000000000001e-06 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233863 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465607+0000", 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.9797999999999996e-05, 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004330403, 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233863 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465607+0000", 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465607+0000", 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465607+0000", 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6600000000000002e-07 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465609+0000", 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8700000000000001e-06 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465614+0000", 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1510000000000003e-06 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465618+0000", 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5829999999999998e-06 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233894 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469165+0000", 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.996532, 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080670799999999995, 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233894 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469165+0000", 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469165+0000", 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469166+0000", 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1900000000000001e-07 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469169+0000", 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.332e-06 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469176+0000", 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0870000000000001e-06 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469181+0000", 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0719999999999999e-06 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233509 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456296+0000", 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0094002370000000005, 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013708669999999999, 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233509 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456296+0000", 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456296+0000", 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456297+0000", 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.46e-07 2024-09-16T13:00:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456299+0000", 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8959999999999999e-06 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456304+0000", 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3669999999999996e-06 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456308+0000", 2024-09-16T13:00:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0600000000000001e-06 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233605 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456857+0000", 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0088397519999999993, 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013193312, 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233605 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456857+0000", 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456857+0000", 2024-09-16T13:00:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456857+0000", 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9699999999999996e-07 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456859+0000", 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8220000000000001e-06 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456865+0000", 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6559999999999997e-06 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456868+0000", 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.331e-06 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233640 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457506+0000", 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0081909059999999995, 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012612249000000001, 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233640 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457506+0000", 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457506+0000", 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457506+0000", 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3700000000000001e-07 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457508+0000", 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7549999999999999e-06 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457513+0000", 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3680000000000001e-06 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457516+0000", 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4230000000000001e-06 2024-09-16T13:00:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233736 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458064+0000", 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0076321490000000004, 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01212149, 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233736 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458064+0000", 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458064+0000", 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458065+0000", 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0499999999999999e-07 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458067+0000", 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1950000000000002e-06 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458072+0000", 2024-09-16T13:00:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4500000000000003e-06 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458076+0000", 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233768 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458879+0000", 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0068177910000000001, 2024-09-16T13:00:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011374716999999999, 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233768 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458879+0000", 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458879+0000", 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458879+0000", 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2200000000000004e-07 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458881+0000", 2024-09-16T13:00:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0499999999999999e-06 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458887+0000", 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4360000000000001e-06 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458890+0000", 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5619999999999999e-06 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233864 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465623+0000", 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 7.3635999999999998e-05, 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047003130000000002, 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233864 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465623+0000", 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465623+0000", 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465623+0000", 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465630+0000", 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1239999999999999e-06 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465636+0000", 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9780000000000002e-06 2024-09-16T13:00:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465640+0000", 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4259999999999999e-06 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233895 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469185+0000", 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.996511, 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011987460000000001, 2024-09-16T13:00:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233895 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469185+0000", 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469185+0000", 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469186+0000", 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4400000000000001e-07 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469188+0000", 2024-09-16T13:00:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3180000000000001e-06 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469194+0000", 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7780000000000004e-06 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469198+0000", 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233510 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456312+0000", 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0093843390000000002, 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014106662000000001, 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233510 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456312+0000", 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456312+0000", 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456313+0000", 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2800000000000003e-07 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456314+0000", 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.86e-06 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456320+0000", 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.395e-06 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456323+0000", 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.354e-06 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233606 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456872+0000", 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0088247129999999997, 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013580115, 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233606 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456872+0000", 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456872+0000", 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456872+0000", 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.08e-07 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456874+0000", 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0619999999999999e-06 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456880+0000", 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6230000000000002e-06 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456883+0000", 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5149999999999998e-06 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233641 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457520+0000", 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0081762399999999995, 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012964926, 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233641 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457520+0000", 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457520+0000", 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457521+0000", 2024-09-16T13:00:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457523+0000", 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0640000000000001e-06 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457528+0000", 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5589999999999996e-06 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457532+0000", 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6100000000000002e-06 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233737 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458079+0000", 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0076175430000000001, 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012442939, 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233737 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458079+0000", 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458079+0000", 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458080+0000", 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8899999999999999e-07 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458082+0000", 2024-09-16T13:00:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8780000000000001e-06 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458087+0000", 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039999999999999e-06 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458092+0000", 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5210000000000004e-06 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233769 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458894+0000", 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0068024019999999999, 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011690818, 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233769 2024-09-16T13:00:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458894+0000", 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458894+0000", 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459639+0000", 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074467100000000003 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459643+0000", 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7629999999999998e-06 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459652+0000", 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0890000000000006e-06 2024-09-16T13:00:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459657+0000", 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2310000000000004e-06 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233865 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465644+0000", 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2967999999999998e-05, 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050099680000000001, 2024-09-16T13:00:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233865 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465644+0000", 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465644+0000", 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465644+0000", 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4900000000000001e-07 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465646+0000", 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9069999999999999e-06 2024-09-16T13:00:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465652+0000", 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.711e-06 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465657+0000", 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1630000000000004e-06 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233896 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469202+0000", 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9964948, 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015202830000000001, 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233896 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469202+0000", 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469202+0000", 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469202+0000", 2024-09-16T13:00:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469204+0000", 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0219999999999999e-06 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469211+0000", 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7440000000000001e-06 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469214+0000", 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7120000000000002e-06 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233511 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456326+0000", 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0093700490000000001, 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014463152, 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233511 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456326+0000", 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456326+0000", 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456327+0000", 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8700000000000001e-07 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456334+0000", 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3549999999999999e-06 2024-09-16T13:00:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456341+0000", 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5300000000000002e-06 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456345+0000", 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8210000000000003e-06 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233607 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456889+0000", 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0088071450000000006, 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013953765999999999, 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233607 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456889+0000", 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456889+0000", 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456890+0000", 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456892+0000", 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9920000000000002e-06 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456897+0000", 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6210000000000001e-06 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456901+0000", 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6320000000000001e-06 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233642 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457536+0000", 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0081609890000000004, 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013341844, 2024-09-16T13:00:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233642 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457536+0000", 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457536+0000", 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457536+0000", 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5499999999999998e-07 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457538+0000", 2024-09-16T13:00:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.866e-06 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457543+0000", 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4399999999999996e-06 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457546+0000", 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233738 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458097+0000", 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0075995749999999999, 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0128142, 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233738 2024-09-16T13:00:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458097+0000", 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458097+0000", 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458097+0000", 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2399999999999998e-07 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458099+0000", 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458105+0000", 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4729999999999999e-06 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458108+0000", 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4139999999999998e-06 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233770 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.459662+0000", 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0060341930000000002, 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011284333000000001, 2024-09-16T13:00:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233770 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459662+0000", 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459662+0000", 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459663+0000", 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9299999999999998e-07 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459665+0000", 2024-09-16T13:00:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0480000000000001e-06 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459671+0000", 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8259999999999998e-06 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459675+0000", 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233866 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465661+0000", 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5716999999999997e-05, 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005350077, 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233866 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465661+0000", 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465661+0000", 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465661+0000", 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9799999999999999e-07 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465663+0000", 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9319999999999999e-06 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465669+0000", 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.609e-06 2024-09-16T13:00:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465674+0000", 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4210000000000003e-06 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233897 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469218+0000", 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9964781, 2024-09-16T13:00:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001861742, 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233897 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469218+0000", 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469218+0000", 2024-09-16T13:00:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469219+0000", 2024-09-16T13:00:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9700000000000002e-07 2024-09-16T13:00:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469221+0000", 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1430000000000001e-06 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469227+0000", 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5419999999999996e-06 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469232+0000", 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3859999999999998e-06 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233512 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456359+0000", 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0093378260000000005, 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014791377, 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233512 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456359+0000", 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456359+0000", 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456359+0000", 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.75e-07 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456361+0000", 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2050000000000001e-06 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456367+0000", 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8860000000000001e-06 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456371+0000", 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5990000000000002e-06 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233608 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456906+0000", 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0087907809999999992, 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014312531999999999, 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233608 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456906+0000", 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456906+0000", 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456906+0000", 2024-09-16T13:00:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0600000000000001e-07 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456908+0000", 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.889e-06 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456914+0000", 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6400000000000002e-06 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456917+0000", 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2909999999999999e-06 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233643 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457563+0000", 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0081337590000000008, 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013724287999999999, 2024-09-16T13:00:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233643 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457563+0000", 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457563+0000", 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457563+0000", 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1200000000000002e-07 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457566+0000", 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3070000000000001e-06 2024-09-16T13:00:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457572+0000", 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4139999999999999e-06 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457576+0000", 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7720000000000001e-06 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233739 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458112+0000", 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0075848130000000001, 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013244605, 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233739 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458112+0000", 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458112+0000", 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458112+0000", 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.65e-07 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458119+0000", 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4880000000000004e-06 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458125+0000", 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9839999999999998e-06 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458128+0000", 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6069999999999999e-06 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233771 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.459680+0000", 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0060162990000000001, 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011717968, 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233771 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459680+0000", 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459680+0000", 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459681+0000", 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9999999999999998e-07 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459682+0000", 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.781e-06 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459688+0000", 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6840000000000001e-06 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459692+0000", 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7919999999999999e-06 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233867 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465678+0000", 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.8493000000000001e-05, 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057531830000000003, 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233867 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465678+0000", 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465678+0000", 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465678+0000", 2024-09-16T13:00:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2199999999999999e-07 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465680+0000", 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.894e-06 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465686+0000", 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6869999999999999e-06 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465690+0000", 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4979999999999998e-06 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233898 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469236+0000", 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9964609, 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022291139999999999, 2024-09-16T13:00:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233898 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469236+0000", 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469236+0000", 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469236+0000", 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4300000000000003e-07 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469238+0000", 2024-09-16T13:00:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469244+0000", 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0190000000000003e-06 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469248+0000", 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.405e-06 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233513 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456375+0000", 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0093218780000000005, 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015125563999999999, 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233513 2024-09-16T13:00:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456375+0000", 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456375+0000", 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456375+0000", 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0299999999999999e-07 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456377+0000", 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.182e-06 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456383+0000", 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456386+0000", 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4130000000000002e-06 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233609 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456923+0000", 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0087739859999999992, 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014610590999999999, 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233609 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456923+0000", 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456923+0000", 2024-09-16T13:00:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456923+0000", 2024-09-16T13:00:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6100000000000002e-07 2024-09-16T13:00:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456925+0000", 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.934e-06 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456932+0000", 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0480000000000001e-06 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456935+0000", 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:00:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233644 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457580+0000", 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0081168520000000008, 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013986215999999999, 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233644 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457580+0000", 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457580+0000", 2024-09-16T13:00:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457580+0000", 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9500000000000003e-07 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457582+0000", 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457587+0000", 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8459999999999999e-06 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457591+0000", 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5190000000000001e-06 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233740 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458132+0000", 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0075644129999999999, 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013473242, 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233740 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458132+0000", 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458132+0000", 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458133+0000", 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9700000000000002e-07 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458134+0000", 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.894e-06 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458140+0000", 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.276e-06 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458143+0000", 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4070000000000002e-06 2024-09-16T13:00:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233772 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.459696+0000", 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0060007359999999996, 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011977952, 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233772 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459696+0000", 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459696+0000", 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459696+0000", 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3799999999999997e-07 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459698+0000", 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9360000000000002e-06 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459704+0000", 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6149999999999996e-06 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459707+0000", 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6100000000000002e-06 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233868 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465706+0000", 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9999905, 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060356020000000002, 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233868 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465706+0000", 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465706+0000", 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465707+0000", 2024-09-16T13:00:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9700000000000005e-07 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465709+0000", 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1330000000000002e-06 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465715+0000", 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2480000000000001e-06 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465719+0000", 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7689999999999998e-06 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233899 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469251+0000", 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9964452, 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025565829999999999, 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233899 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469251+0000", 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469251+0000", 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469252+0000", 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7e-07 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469254+0000", 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0150000000000002e-06 2024-09-16T13:00:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469259+0000", 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469264+0000", 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4429999999999996e-06 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233514 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456389+0000", 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0093070540000000004, 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015481089, 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233514 2024-09-16T13:00:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456389+0000", 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456389+0000", 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456390+0000", 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5600000000000001e-07 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456392+0000", 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8059999999999999e-06 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456397+0000", 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.417e-06 2024-09-16T13:00:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456403+0000", 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4420000000000003e-06 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233610 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456939+0000", 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0087573660000000008, 2024-09-16T13:00:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014992343, 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233610 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456939+0000", 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456939+0000", 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456940+0000", 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5499999999999998e-07 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456942+0000", 2024-09-16T13:00:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.897e-06 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456947+0000", 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6060000000000002e-06 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456951+0000", 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5439999999999999e-06 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233645 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457595+0000", 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0081015519999999997, 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014403851000000001, 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233645 2024-09-16T13:00:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457595+0000", 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457595+0000", 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457596+0000", 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7599999999999997e-07 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457597+0000", 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.911e-06 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457603+0000", 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4859999999999997e-06 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457606+0000", 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3519999999999998e-06 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233741 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458147+0000", 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0075491179999999996, 2024-09-16T13:00:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013918521, 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233741 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458147+0000", 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458147+0000", 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458148+0000", 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458150+0000", 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.875e-06 2024-09-16T13:00:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458156+0000", 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1279999999999996e-06 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458160+0000", 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3409999999999996e-06 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233773 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.459711+0000", 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0059853800000000002, 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012420260000000001, 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233773 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459711+0000", 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459711+0000", 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459712+0000", 2024-09-16T13:00:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.08e-07 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459713+0000", 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9039999999999999e-06 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459719+0000", 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4129999999999997e-06 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459723+0000", 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7419999999999999e-06 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233869 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465723+0000", 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9999738, 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064773340000000004, 2024-09-16T13:00:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233869 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465723+0000", 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465723+0000", 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465723+0000", 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5600000000000005e-07 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465725+0000", 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9139999999999998e-06 2024-09-16T13:00:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465730+0000", 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1089999999999997e-06 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465734+0000", 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4829999999999999e-06 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233900 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469267+0000", 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.996429, 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030097029999999999, 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233900 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469267+0000", 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469267+0000", 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469268+0000", 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0200000000000001e-07 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469275+0000", 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9609999999999999e-06 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469281+0000", 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5090000000000001e-06 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469284+0000", 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5149999999999998e-06 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233515 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456407+0000", 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0092892219999999998, 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015943043, 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233515 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456407+0000", 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456407+0000", 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456408+0000", 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456410+0000", 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0889999999999998e-06 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456415+0000", 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5690000000000004e-06 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456419+0000", 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2550000000000002e-06 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233611 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456955+0000", 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0087419630000000002, 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015458705, 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233611 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456955+0000", 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456955+0000", 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456955+0000", 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0999999999999999e-07 2024-09-16T13:00:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456961+0000", 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9070000000000004e-06 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456967+0000", 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7440000000000004e-06 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456970+0000", 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3189999999999999e-06 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233646 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457610+0000", 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0080866889999999993, 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014866251, 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233646 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457610+0000", 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457610+0000", 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457610+0000", 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457612+0000", 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0420000000000001e-06 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457618+0000", 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4600000000000002e-06 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457622+0000", 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1509999999999997e-06 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233742 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458164+0000", 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0075326029999999997, 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014379281000000001, 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233742 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458164+0000", 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458164+0000", 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458164+0000", 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.15e-07 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458167+0000", 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.215e-06 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458172+0000", 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4269999999999999e-06 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458175+0000", 2024-09-16T13:00:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.393e-06 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233774 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.459736+0000", 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0059602149999999996, 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012884525000000001, 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233774 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459736+0000", 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459736+0000", 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459737+0000", 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459740+0000", 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6230000000000001e-06 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459746+0000", 2024-09-16T13:00:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6819999999999997e-06 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459750+0000", 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.963e-06 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233870 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465738+0000", 2024-09-16T13:00:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.999959, 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069326479999999996, 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233870 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465738+0000", 2024-09-16T13:00:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465738+0000", 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465738+0000", 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465740+0000", 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2290000000000002e-06 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465746+0000", 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.541e-06 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465749+0000", 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.636e-06 2024-09-16T13:00:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233901 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469289+0000", 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9964075, 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034161360000000002, 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233901 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469289+0000", 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469289+0000", 2024-09-16T13:00:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469289+0000", 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9599999999999999e-07 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469291+0000", 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.877e-06 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469297+0000", 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4709999999999998e-06 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469300+0000", 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.467e-06 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233516 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456423+0000", 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0092739239999999994, 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016337382000000001, 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233516 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456423+0000", 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456423+0000", 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456423+0000", 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:00:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456425+0000", 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9149999999999999e-06 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456430+0000", 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4060000000000004e-06 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456434+0000", 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3510000000000002e-06 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233612 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456974+0000", 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0087226839999999996, 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015824993999999998, 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233612 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456974+0000", 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456974+0000", 2024-09-16T13:00:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456974+0000", 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1100000000000001e-07 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456976+0000", 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0389999999999999e-06 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456982+0000", 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6919999999999999e-06 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456985+0000", 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4599999999999999e-06 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233647 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457625+0000", 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0080712930000000002, 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015228993, 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233647 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457625+0000", 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457625+0000", 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457626+0000", 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:00:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457627+0000", 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8989999999999999e-06 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457633+0000", 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9889999999999997e-06 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457638+0000", 2024-09-16T13:00:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3000000000000003e-06 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233743 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458179+0000", 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0075177589999999997, 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014716971000000001, 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233743 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458179+0000", 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458179+0000", 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458179+0000", 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8000000000000001e-07 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458181+0000", 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.773e-06 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458186+0000", 2024-09-16T13:00:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4739999999999996e-06 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458191+0000", 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7849999999999999e-06 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233775 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.459754+0000", 2024-09-16T13:00:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0059424329999999996, 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013179417000000001, 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233775 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459754+0000", 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459754+0000", 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459755+0000", 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459761+0000", 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4949999999999996e-06 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459767+0000", 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9100000000000002e-06 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459771+0000", 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.72e-06 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233871 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465755+0000", 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9999418, 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072221250000000002, 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233871 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465755+0000", 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465755+0000", 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465755+0000", 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:00:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465757+0000", 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8929999999999999e-06 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465765+0000", 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7959999999999996e-06 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465769+0000", 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2400000000000001e-06 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233902 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469308+0000", 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9963889, 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037091509999999999, 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233902 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469308+0000", 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469308+0000", 2024-09-16T13:00:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469308+0000", 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6600000000000002e-07 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469310+0000", 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9199999999999998e-06 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469315+0000", 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3920000000000002e-06 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469319+0000", 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5640000000000001e-06 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233517 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456437+0000", 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0092594449999999998, 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016648102000000001, 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233517 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456437+0000", 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456437+0000", 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456437+0000", 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3599999999999999e-07 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456439+0000", 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8610000000000001e-06 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456445+0000", 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5269999999999998e-06 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456448+0000", 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3850000000000002e-06 2024-09-16T13:00:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233613 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456991+0000", 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0087056819999999993, 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016161411000000001, 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233613 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456991+0000", 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456991+0000", 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456991+0000", 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.22e-07 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456993+0000", 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.795e-06 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456999+0000", 2024-09-16T13:00:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7830000000000004e-06 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457002+0000", 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7050000000000001e-06 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233648 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457642+0000", 2024-09-16T13:00:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0080548430000000008, 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015553905999999999, 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233648 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457642+0000", 2024-09-16T13:00:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457642+0000", 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457642+0000", 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9400000000000001e-07 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457644+0000", 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9740000000000001e-06 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457650+0000", 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1330000000000004e-06 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457654+0000", 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4970000000000001e-06 2024-09-16T13:00:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233744 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458195+0000", 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0075015819999999997, 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015034185, 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233744 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458195+0000", 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458195+0000", 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458195+0000", 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6100000000000002e-07 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458198+0000", 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2220000000000001e-06 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458203+0000", 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458207+0000", 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5109999999999999e-06 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233776 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.459775+0000", 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0059219809999999998, 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013487336000000001, 2024-09-16T13:00:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233776 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459775+0000", 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459775+0000", 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459775+0000", 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.15e-07 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459777+0000", 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0140000000000001e-06 2024-09-16T13:00:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459783+0000", 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5470000000000004e-06 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459786+0000", 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2780000000000002e-06 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233872 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465773+0000", 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9999232, 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075230160000000004, 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233872 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465773+0000", 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465773+0000", 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465774+0000", 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4099999999999999e-07 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465776+0000", 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9999999999999999e-06 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465782+0000", 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8480000000000003e-06 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465786+0000", 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.67e-06 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233903 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469323+0000", 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9963741, 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040071309999999997, 2024-09-16T13:00:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233903 2024-09-16T13:00:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469323+0000", 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469323+0000", 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469323+0000", 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6100000000000002e-07 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469325+0000", 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.03e-06 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469331+0000", 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5660000000000003e-06 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469335+0000", 2024-09-16T13:00:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6200000000000001e-06 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233518 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456452+0000", 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0092448389999999995, 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016946822, 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233518 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456452+0000", 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456452+0000", 2024-09-16T13:00:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456452+0000", 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456454+0000", 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456459+0000", 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3279999999999996e-06 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456463+0000", 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5140000000000002e-06 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233614 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457006+0000", 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0086901510000000001, 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016426663000000001, 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233614 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457006+0000", 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457006+0000", 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457007+0000", 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.08e-07 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457009+0000", 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8899999999999999e-06 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457014+0000", 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7350000000000001e-06 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457019+0000", 2024-09-16T13:00:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9300000000000002e-06 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233649 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457659+0000", 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0080378370000000008, 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015822378000000002, 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233649 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457659+0000", 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457659+0000", 2024-09-16T13:00:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457659+0000", 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6800000000000001e-07 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457661+0000", 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9649999999999998e-06 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457668+0000", 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.323e-06 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457672+0000", 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233745 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458210+0000", 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.007486398, 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015305361, 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233745 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458210+0000", 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458210+0000", 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458210+0000", 2024-09-16T13:00:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5499999999999999e-07 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458212+0000", 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8080000000000001e-06 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458218+0000", 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7769999999999999e-06 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458221+0000", 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2729999999999998e-06 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233777 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.459790+0000", 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.005906929, 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013760494, 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233777 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459790+0000", 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.459790+0000", 2024-09-16T13:00:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460822+0000", 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001032493 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460826+0000", 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7189999999999999e-06 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460834+0000", 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6270000000000006e-06 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460839+0000", 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9910000000000002e-06 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233873 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465790+0000", 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.999907, 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077946439999999999, 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233873 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465790+0000", 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465790+0000", 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465790+0000", 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1100000000000001e-07 2024-09-16T13:00:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465792+0000", 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.792e-06 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465797+0000", 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6300000000000003e-06 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465801+0000", 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233904 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469339+0000", 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9963574, 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042793800000000002, 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233904 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469339+0000", 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469339+0000", 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469339+0000", 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1199999999999998e-07 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469342+0000", 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469348+0000", 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2380000000000002e-06 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469351+0000", 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7050000000000001e-06 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233519 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456467+0000", 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0092297990000000003, 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017186514, 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233519 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456467+0000", 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456467+0000", 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456467+0000", 2024-09-16T13:00:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6800000000000001e-07 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456469+0000", 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0420000000000001e-06 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456475+0000", 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6019999999999999e-06 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456478+0000", 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3050000000000001e-06 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233615 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457023+0000", 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0086735279999999998, 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016664221999999999, 2024-09-16T13:00:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233615 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457023+0000", 2024-09-16T13:00:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457023+0000", 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457023+0000", 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7099999999999997e-07 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457025+0000", 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.99e-06 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457031+0000", 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9000000000000003e-06 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457035+0000", 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5300000000000001e-06 2024-09-16T13:00:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233650 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457676+0000", 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0080208399999999996, 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016044835, 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233650 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457676+0000", 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457676+0000", 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457676+0000", 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0699999999999998e-07 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457678+0000", 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.824e-06 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457683+0000", 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3210000000000004e-06 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457686+0000", 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.123e-06 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233746 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458226+0000", 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0074704769999999997, 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015530733, 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233746 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458226+0000", 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458226+0000", 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458227+0000", 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2799999999999996e-07 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458229+0000", 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0099999999999998e-06 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458235+0000", 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8560000000000003e-06 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458238+0000", 2024-09-16T13:00:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4580000000000002e-06 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233778 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.460844+0000", 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0048520300000000002, 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012945262000000001, 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233778 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460844+0000", 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460844+0000", 2024-09-16T13:00:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460845+0000", 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7599999999999997e-07 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460847+0000", 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3740000000000001e-06 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460853+0000", 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1070000000000001e-06 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460857+0000", 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6890000000000002e-06 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233874 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465805+0000", 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9998913, 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080181799999999998, 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233874 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465805+0000", 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465805+0000", 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465805+0000", 2024-09-16T13:00:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9000000000000002e-07 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465808+0000", 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1390000000000002e-06 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465813+0000", 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4759999999999997e-06 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465816+0000", 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3409999999999999e-06 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233905 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469355+0000", 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9963417, 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045024110000000004, 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233905 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469355+0000", 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469355+0000", 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469355+0000", 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469357+0000", 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.933e-06 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469362+0000", 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.378e-06 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469366+0000", 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5250000000000001e-06 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233520 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456482+0000", 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0092149329999999998, 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017411308, 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233520 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456482+0000", 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456482+0000", 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456482+0000", 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2200000000000004e-07 2024-09-16T13:00:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456484+0000", 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8300000000000001e-06 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456489+0000", 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3090000000000003e-06 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456493+0000", 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3110000000000001e-06 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233616 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457039+0000", 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0086579619999999999, 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016888100999999999, 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233616 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457039+0000", 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457039+0000", 2024-09-16T13:00:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457039+0000", 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8099999999999998e-07 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457041+0000", 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1909999999999999e-06 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457048+0000", 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6340000000000003e-06 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457051+0000", 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.366e-06 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233651 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457693+0000", 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0080031010000000003, 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016266552, 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233651 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457693+0000", 2024-09-16T13:00:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457693+0000", 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457694+0000", 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457696+0000", 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9599999999999999e-06 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457702+0000", 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1589999999999998e-06 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457705+0000", 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4979999999999998e-06 2024-09-16T13:00:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233747 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458242+0000", 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0074547820000000001, 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015751477999999999, 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233747 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458242+0000", 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458242+0000", 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458242+0000", 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8400000000000005e-07 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458244+0000", 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8980000000000001e-06 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458249+0000", 2024-09-16T13:00:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3299999999999998e-06 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458253+0000", 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1889999999999999e-06 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233779 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.460861+0000", 2024-09-16T13:00:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0048354540000000003, 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013165637000000001, 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233779 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460861+0000", 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460861+0000", 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460861+0000", 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.72e-07 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460863+0000", 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.945e-06 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460869+0000", 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4299999999999997e-06 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460872+0000", 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4510000000000001e-06 2024-09-16T13:00:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233875 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465821+0000", 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9998755, 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082399719999999999, 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233875 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465821+0000", 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465821+0000", 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465821+0000", 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7599999999999998e-07 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465823+0000", 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8020000000000001e-06 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465828+0000", 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3759999999999999e-06 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465832+0000", 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3270000000000001e-06 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233906 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469371+0000", 2024-09-16T13:00:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.996325, 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047231599999999997, 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233906 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469371+0000", 2024-09-16T13:00:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469371+0000", 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469372+0000", 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3500000000000002e-07 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469374+0000", 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.916e-06 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469379+0000", 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469383+0000", 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1629999999999998e-06 2024-09-16T13:00:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233521 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456506+0000", 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0091906490000000004, 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017624419999999998, 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233521 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456506+0000", 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456506+0000", 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456506+0000", 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2800000000000002e-07 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456508+0000", 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.962e-06 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456514+0000", 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0190000000000003e-06 2024-09-16T13:00:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456518+0000", 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.602e-06 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233617 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457057+0000", 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0086399179999999999, 2024-09-16T13:00:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017106731, 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233617 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457057+0000", 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457057+0000", 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457057+0000", 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.72e-07 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457059+0000", 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8959999999999999e-06 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457065+0000", 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7200000000000003e-06 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457068+0000", 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3579999999999999e-06 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233652 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457709+0000", 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0079872380000000007, 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016487517, 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233652 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457709+0000", 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457709+0000", 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457710+0000", 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.44e-07 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457712+0000", 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9889999999999999e-06 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457718+0000", 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3949999999999998e-06 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457722+0000", 2024-09-16T13:00:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5829999999999998e-06 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233748 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458256+0000", 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.00744037, 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015973550999999999, 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233748 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458256+0000", 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458256+0000", 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458257+0000", 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458258+0000", 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-06 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458264+0000", 2024-09-16T13:00:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.259e-06 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458267+0000", 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3629999999999998e-06 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233780 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.460876+0000", 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0048201310000000001, 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013386356, 2024-09-16T13:00:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233780 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460876+0000", 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460876+0000", 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460877+0000", 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9000000000000002e-07 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460879+0000", 2024-09-16T13:00:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.057e-06 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460885+0000", 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7860000000000002e-06 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460888+0000", 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5030000000000002e-06 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233876 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465835+0000", 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9998612, 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084618430000000001, 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233876 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465835+0000", 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465835+0000", 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465836+0000", 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465842+0000", 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6200000000000001e-06 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465848+0000", 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0580000000000003e-06 2024-09-16T13:00:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465854+0000", 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2619999999999998e-06 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233907 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469387+0000", 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9963098, 2024-09-16T13:00:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049435110000000003, 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233907 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469387+0000", 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469387+0000", 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469387+0000", 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6899999999999997e-07 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469390+0000", 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.249e-06 2024-09-16T13:00:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469396+0000", 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2060000000000004e-06 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469400+0000", 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3819999999999997e-06 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233522 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456522+0000", 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0091750070000000006, 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017843774, 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233522 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456522+0000", 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456522+0000", 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456522+0000", 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6400000000000003e-07 2024-09-16T13:00:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456524+0000", 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9769999999999999e-06 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456531+0000", 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2520000000000002e-06 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456535+0000", 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7139999999999999e-06 2024-09-16T13:00:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233618 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457072+0000", 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0086248469999999997, 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017326463, 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233618 2024-09-16T13:00:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457072+0000", 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457072+0000", 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457072+0000", 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1500000000000004e-07 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457074+0000", 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.871e-06 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457080+0000", 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.942e-06 2024-09-16T13:00:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457083+0000", 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3950000000000001e-06 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233653 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457734+0000", 2024-09-16T13:00:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0079627840000000005, 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016698582999999999, 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233653 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457734+0000", 2024-09-16T13:00:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457734+0000", 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457734+0000", 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9200000000000002e-07 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457736+0000", 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1789999999999998e-06 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457743+0000", 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1620000000000002e-06 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457747+0000", 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8099999999999999e-06 2024-09-16T13:00:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233749 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458270+0000", 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0074260690000000004, 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016194692, 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233749 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458270+0000", 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458270+0000", 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458271+0000", 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2699999999999999e-07 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458273+0000", 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9630000000000001e-06 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458279+0000", 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9179999999999999e-06 2024-09-16T13:00:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458283+0000", 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9090000000000002e-06 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233781 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.460891+0000", 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.00480504, 2024-09-16T13:00:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013606617999999999, 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233781 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460891+0000", 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460891+0000", 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460892+0000", 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460894+0000", 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9479999999999998e-06 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460899+0000", 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6790000000000002e-06 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460903+0000", 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4319999999999999e-06 2024-09-16T13:00:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233877 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465857+0000", 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9998393, 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086739380000000008, 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233877 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465857+0000", 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465857+0000", 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465858+0000", 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465860+0000", 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0250000000000001e-06 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465865+0000", 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2669999999999997e-06 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465869+0000", 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4699999999999998e-06 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233908 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469408+0000", 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9962888, 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005158017, 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233908 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469408+0000", 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469408+0000", 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469408+0000", 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7000000000000004e-07 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469410+0000", 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9069999999999999e-06 2024-09-16T13:00:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469416+0000", 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6690000000000003e-06 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469420+0000", 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4800000000000001e-06 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233523 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456539+0000", 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0091577770000000006, 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018063735000000001, 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233523 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456539+0000", 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456539+0000", 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456539+0000", 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9999999999999998e-07 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456545+0000", 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7019999999999998e-06 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456550+0000", 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6409999999999999e-06 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456554+0000", 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4199999999999999e-06 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233619 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457094+0000", 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0086027080000000006, 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017541859999999999, 2024-09-16T13:00:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233619 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457094+0000", 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457094+0000", 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457094+0000", 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8299999999999998e-07 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457096+0000", 2024-09-16T13:00:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1280000000000002e-06 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457104+0000", 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4320000000000002e-06 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457108+0000", 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.912e-06 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233654 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457752+0000", 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0079448820000000003, 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016916839999999999, 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233654 2024-09-16T13:00:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457752+0000", 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457752+0000", 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457752+0000", 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1099999999999996e-07 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457754+0000", 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9640000000000002e-06 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457760+0000", 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5300000000000004e-06 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457763+0000", 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233750 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458289+0000", 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0074079230000000003, 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016414232000000001, 2024-09-16T13:00:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233750 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458289+0000", 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458289+0000", 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458289+0000", 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0999999999999999e-07 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458291+0000", 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.001e-06 2024-09-16T13:00:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458297+0000", 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7230000000000001e-06 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458300+0000", 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3330000000000001e-06 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233782 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.460906+0000", 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0047900299999999998, 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013830461, 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233782 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460906+0000", 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460906+0000", 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460907+0000", 2024-09-16T13:00:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8700000000000001e-07 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460909+0000", 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1780000000000002e-06 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460914+0000", 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.237e-06 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460918+0000", 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9389999999999999e-06 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233878 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465900+0000", 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9997964, 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0088700540000000005, 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233878 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465900+0000", 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465900+0000", 2024-09-16T13:00:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465901+0000", 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5099999999999999e-07 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465903+0000", 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.193e-06 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465910+0000", 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4999999999999996e-06 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465914+0000", 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8210000000000003e-06 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233909 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469423+0000", 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.996273, 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053806130000000002, 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233909 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469423+0000", 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469423+0000", 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469424+0000", 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7399999999999998e-07 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469426+0000", 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9319999999999999e-06 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469431+0000", 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3229999999999997e-06 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469434+0000", 2024-09-16T13:00:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2200000000000001e-06 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233524 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456557+0000", 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0091390599999999992, 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018281209999999999, 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233524 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456557+0000", 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456557+0000", 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456558+0000", 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8799999999999998e-07 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456560+0000", 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7740000000000001e-06 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456565+0000", 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4389999999999999e-06 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456568+0000", 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233620 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457111+0000", 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0085850800000000001, 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017760096999999999, 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233620 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457111+0000", 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457111+0000", 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457112+0000", 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8999999999999997e-07 2024-09-16T13:00:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457114+0000", 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9240000000000001e-06 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457120+0000", 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2090000000000002e-06 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457130+0000", 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7750000000000004e-06 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233655 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457766+0000", 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0079300350000000002, 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017139647000000001, 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233655 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457766+0000", 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457766+0000", 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457767+0000", 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457769+0000", 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.809e-06 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457774+0000", 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457780+0000", 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6659999999999996e-06 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233751 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458304+0000", 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0073930080000000004, 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016637662000000001, 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233751 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458304+0000", 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458304+0000", 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458304+0000", 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7300000000000002e-07 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458309+0000", 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5040000000000002e-06 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458315+0000", 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.66e-06 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458318+0000", 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1489999999999998e-06 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233783 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.460922+0000", 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0047745289999999996, 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014052944, 2024-09-16T13:00:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233783 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460922+0000", 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460922+0000", 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460922+0000", 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7300000000000002e-07 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460924+0000", 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1299999999999999e-06 2024-09-16T13:00:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460930+0000", 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4620000000000004e-06 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460934+0000", 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.709e-06 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233879 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465918+0000", 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9997783, 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0090918319999999993, 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233879 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465918+0000", 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465918+0000", 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465919+0000", 2024-09-16T13:00:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7e-07 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465921+0000", 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465926+0000", 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2310000000000004e-06 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465929+0000", 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4020000000000002e-06 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233910 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469438+0000", 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9962587, 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005605957, 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233910 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469438+0000", 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469438+0000", 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469438+0000", 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.34e-07 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469440+0000", 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9230000000000001e-06 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469447+0000", 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.464e-06 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469451+0000", 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6289999999999999e-06 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233525 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456572+0000", 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0091242939999999998, 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018508365999999998, 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233525 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456572+0000", 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456572+0000", 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456573+0000", 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0600000000000001e-07 2024-09-16T13:00:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456574+0000", 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.832e-06 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456580+0000", 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9449999999999998e-06 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456584+0000", 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3560000000000001e-06 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233621 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457135+0000", 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0085615889999999997, 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017979115, 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233621 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457135+0000", 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457135+0000", 2024-09-16T13:00:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457135+0000", 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5299999999999999e-07 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457138+0000", 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2220000000000001e-06 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457143+0000", 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6969999999999998e-06 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457147+0000", 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.399e-06 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233656 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457784+0000", 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0079129779999999993, 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017365080000000001, 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233656 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457784+0000", 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457784+0000", 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457784+0000", 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2099999999999996e-07 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457786+0000", 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0339999999999999e-06 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457792+0000", 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6200000000000004e-06 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457795+0000", 2024-09-16T13:00:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233752 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458322+0000", 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0073746189999999998, 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016859395999999999, 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233752 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458322+0000", 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458322+0000", 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458322+0000", 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.72e-07 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458324+0000", 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7850000000000001e-06 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458329+0000", 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3009999999999997e-06 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458332+0000", 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1039999999999998e-06 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233784 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.460937+0000", 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0047594869999999997, 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01427725, 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233784 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460937+0000", 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460937+0000", 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460937+0000", 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9499999999999998e-07 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460939+0000", 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.869e-06 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460945+0000", 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8270000000000003e-06 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460949+0000", 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4350000000000001e-06 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233880 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465934+0000", 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.999763, 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093151550000000003, 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233880 2024-09-16T13:00:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465934+0000", 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465934+0000", 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465934+0000", 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000001e-07 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465936+0000", 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9439999999999999e-06 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465942+0000", 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6529999999999999e-06 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465945+0000", 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4180000000000001e-06 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233911 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469462+0000", 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9962349, 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058202920000000003, 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233911 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469462+0000", 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469462+0000", 2024-09-16T13:00:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469462+0000", 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4700000000000002e-07 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469464+0000", 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1840000000000002e-06 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469470+0000", 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9039999999999997e-06 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469474+0000", 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5860000000000001e-06 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233526 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456587+0000", 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0091090340000000002, 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018730032000000001, 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233526 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456587+0000", 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456587+0000", 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456588+0000", 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.41e-07 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456590+0000", 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9319999999999999e-06 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456595+0000", 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0900000000000004e-06 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456598+0000", 2024-09-16T13:00:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233622 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457151+0000", 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0085458300000000008, 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018199967000000001, 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233622 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457151+0000", 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457151+0000", 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457151+0000", 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8700000000000001e-07 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457153+0000", 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.984e-06 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457160+0000", 2024-09-16T13:00:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4849999999999997e-06 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457164+0000", 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4839999999999998e-06 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233657 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457799+0000", 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0078977619999999991, 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017585773999999998, 2024-09-16T13:00:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233657 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457799+0000", 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457799+0000", 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457799+0000", 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457801+0000", 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.962e-06 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457807+0000", 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3630000000000002e-06 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457810+0000", 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.478e-06 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233753 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458336+0000", 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0073604200000000003, 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017080998, 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233753 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458336+0000", 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458336+0000", 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458336+0000", 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3000000000000002e-07 2024-09-16T13:00:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458338+0000", 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9640000000000002e-06 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458346+0000", 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6760000000000007e-06 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458350+0000", 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4699999999999998e-06 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233785 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.460971+0000", 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0047258839999999996, 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01447906, 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233785 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460971+0000", 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460971+0000", 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460971+0000", 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9100000000000003e-07 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460974+0000", 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3429999999999998e-06 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460980+0000", 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3849999999999999e-06 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460984+0000", 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6320000000000001e-06 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233881 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465949+0000", 2024-09-16T13:00:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9997478, 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095345699999999992, 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233881 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465949+0000", 2024-09-16T13:00:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465949+0000", 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465949+0000", 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9700000000000002e-07 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465951+0000", 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8589999999999999e-06 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465957+0000", 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5629999999999999e-06 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465960+0000", 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4429999999999999e-06 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233912 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469477+0000", 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9962192, 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060396629999999998, 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233912 2024-09-16T13:00:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469477+0000", 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469477+0000", 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469478+0000", 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6100000000000002e-07 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469485+0000", 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6590000000000007e-06 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469491+0000", 2024-09-16T13:00:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.541e-06 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469494+0000", 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5379999999999998e-06 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233527 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456602+0000", 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0090949150000000003, 2024-09-16T13:00:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018949602999999999, 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233527 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456602+0000", 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456602+0000", 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456602+0000", 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7e-07 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456604+0000", 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.832e-06 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456609+0000", 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3539999999999999e-06 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456613+0000", 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.512e-06 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233623 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457240+0000", 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0084568400000000002, 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01834496, 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233623 2024-09-16T13:00:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457240+0000", 2024-09-16T13:00:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457240+0000", 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457240+0000", 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7299999999999995e-07 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457246+0000", 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1489999999999999e-06 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457254+0000", 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999997e-06 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457258+0000", 2024-09-16T13:00:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4449999999999998e-06 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233658 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457815+0000", 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0078814369999999998, 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017802348999999999, 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233658 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457815+0000", 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457815+0000", 2024-09-16T13:00:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457815+0000", 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.39e-07 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457817+0000", 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.894e-06 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457823+0000", 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1410000000000002e-06 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457827+0000", 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4529999999999998e-06 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233754 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458353+0000", 2024-09-16T13:00:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0073432289999999997, 2024-09-16T13:00:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017296879000000001, 2024-09-16T13:00:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233754 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458353+0000", 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458353+0000", 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458354+0000", 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9499999999999998e-07 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458356+0000", 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.872e-06 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458361+0000", 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4840000000000003e-06 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458365+0000", 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4999999999999999e-06 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233786 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.460988+0000", 2024-09-16T13:00:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0047087759999999996, 2024-09-16T13:00:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01469557, 2024-09-16T13:00:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233786 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460988+0000", 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460988+0000", 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460988+0000", 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5499999999999998e-07 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460990+0000", 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9319999999999999e-06 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460996+0000", 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4009999999999996e-06 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.460999+0000", 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4079999999999998e-06 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233882 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465964+0000", 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9997325, 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097694649999999997, 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233882 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465964+0000", 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465964+0000", 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465964+0000", 2024-09-16T13:00:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9999999999999998e-07 2024-09-16T13:00:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465966+0000", 2024-09-16T13:00:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9309999999999998e-06 2024-09-16T13:00:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465974+0000", 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1659999999999994e-06 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465978+0000", 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6160000000000002e-06 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233913 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469499+0000", 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9961977, 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063009889999999999, 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233913 2024-09-16T13:00:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469499+0000", 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469499+0000", 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469499+0000", 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6300000000000001e-07 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469501+0000", 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0760000000000001e-06 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469507+0000", 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469510+0000", 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.337e-06 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233528 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456617+0000", 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0090799509999999993, 2024-09-16T13:00:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019254006000000001, 2024-09-16T13:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233528 2024-09-16T13:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456617+0000", 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456617+0000", 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456617+0000", 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0400000000000002e-07 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456619+0000", 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9889999999999999e-06 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456625+0000", 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6030000000000004e-06 2024-09-16T13:00:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456628+0000", 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233624 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457262+0000", 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0084343319999999992, 2024-09-16T13:00:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018675112000000001, 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233624 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457262+0000", 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457262+0000", 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457263+0000", 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9999999999999997e-07 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457265+0000", 2024-09-16T13:00:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-16T13:00:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457271+0000", 2024-09-16T13:00:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6409999999999999e-06 2024-09-16T13:00:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457274+0000", 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233659 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457831+0000", 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0078659860000000002, 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018174078, 2024-09-16T13:00:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233659 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457831+0000", 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457831+0000", 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457831+0000", 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7300000000000002e-07 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457833+0000", 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9580000000000001e-06 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457838+0000", 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1880000000000001e-06 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457841+0000", 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1669999999999999e-06 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233755 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458368+0000", 2024-09-16T13:00:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0073286380000000002, 2024-09-16T13:00:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017704188999999999, 2024-09-16T13:00:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233755 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458368+0000", 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458368+0000", 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458368+0000", 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1200000000000003e-07 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458370+0000", 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8199999999999999e-06 2024-09-16T13:00:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458376+0000", 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.429e-06 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458379+0000", 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.224e-06 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233787 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.461003+0000", 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0046939959999999998, 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015136471, 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233787 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461003+0000", 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461003+0000", 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461138+0000", 2024-09-16T13:00:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135463 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461146+0000", 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1950000000000003e-06 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461154+0000", 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8040000000000002e-06 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461159+0000", 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6909999999999996e-06 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233883 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465982+0000", 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9997144, 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010224608, 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233883 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465982+0000", 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465982+0000", 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465982+0000", 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465984+0000", 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.995e-06 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465990+0000", 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4190000000000001e-06 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465993+0000", 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3890000000000001e-06 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233914 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469514+0000", 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9961824, 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006747744, 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233914 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469514+0000", 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469514+0000", 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469515+0000", 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.08e-07 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469517+0000", 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.934e-06 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469523+0000", 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2920000000000001e-06 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469526+0000", 2024-09-16T13:00:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4640000000000002e-06 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233529 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.456653+0000", 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0090431599999999997, 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019645902999999999, 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233529 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456653+0000", 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456653+0000", 2024-09-16T13:00:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456654+0000", 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1299999999999999e-07 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456656+0000", 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2029999999999999e-06 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456664+0000", 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5669999999999997e-06 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.456668+0000", 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8070000000000001e-06 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233625 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457278+0000", 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0084187059999999998, 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019056061999999999, 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233625 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457278+0000", 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457278+0000", 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457278+0000", 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457280+0000", 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457286+0000", 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6210000000000001e-06 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457290+0000", 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4229999999999998e-06 2024-09-16T13:00:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233660 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.457845+0000", 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0078517929999999993, 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018526763000000002, 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233660 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457845+0000", 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457845+0000", 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457845+0000", 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457847+0000", 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.711e-06 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457854+0000", 2024-09-16T13:00:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7560000000000002e-06 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.457857+0000", 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.264e-06 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233756 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.458382+0000", 2024-09-16T13:00:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0073140560000000002, 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018051391, 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233756 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458382+0000", 2024-09-16T13:00:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458382+0000", 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458383+0000", 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458385+0000", 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.809e-06 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458390+0000", 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.254e-06 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.458393+0000", 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3610000000000001e-06 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233788 4.13 4.3cde4433 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.461164+0000", 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0045329209999999997, 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015340769000000001, 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233788 2024-09-16T13:00:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461164+0000", 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461164+0000", 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461252+0000", 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.844e-05 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461255+0000", 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3210000000000001e-06 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461262+0000", 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1369999999999996e-06 2024-09-16T13:00:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.461267+0000", 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6040000000000003e-06 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233884 4.18 4.5d5cc478 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.465997+0000", 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9996996, 2024-09-16T13:00:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010575476, 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233884 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465997+0000", 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465997+0000", 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465997+0000", 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.465999+0000", 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.841e-06 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.466005+0000", 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4199999999999998e-06 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.466008+0000", 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7230000000000002e-06 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:233915 4.13 4.3cde4433 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.469530+0000", 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4294967295.9961667, 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071101790000000003, 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 233915 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469530+0000", 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469530+0000", 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469531+0000", 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4700000000000001e-07 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469533+0000", 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.968e-06 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469538+0000", 2024-09-16T13:00:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.327e-06 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.469541+0000", 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.287e-06 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 222 2024-09-16T13:00:38.269 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:38.272 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:38.272 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:38.265+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:38.274 INFO:teuthology.orchestra.run.smithi042.stdout:4969 trunc from 0xe300fbc to 0x8b80694 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4972 trunc from 0x8b80694 to 0x5f1a51c 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4974 write 0x7e3d542 thru 0x7e3eb0a (0x15c9 bytes) 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4975 read 0x3e82da7 thru 0x3e8c497 (0x96f1 bytes) 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4976 writesame 0x13233e5 thru 0x1328414 data_size 0x8(0x5030 bytes) 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4978 punch from 0x2be0eae to 0x2be5530, (0x4682 bytes) 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4979 punch from 0x6c70c1e to 0x6c74156, (0x3538 bytes) 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4980 read 0x7a0cf83 thru 0x7a0e328 (0x13a6 bytes) 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4981 punch from 0x2f413ad to 0x2f45c44, (0x4897 bytes) 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4984 trunc from 0x7e3eb0b to 0x2d5f94e 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4985 read 0x464a43 thru 0x466516 (0x1ad4 bytes) 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4986 read 0x178eed thru 0x179041 (0x155 bytes) 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4987 read 0x1433c6 thru 0x14ac4f (0x788a bytes) 2024-09-16T13:00:38.275 INFO:teuthology.orchestra.run.smithi042.stdout:4988 writesame 0x2fa49af thru 0x2faa12d data_size 0x1(0x577f bytes) 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:4989 read 0x206fe3e thru 0x2071918 (0x1adb bytes) 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:4991 read 0x2120f95 thru 0x21226ad (0x1719 bytes) 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:4997 read 0x16d900e thru 0x16e1841 (0x8834 bytes) 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:4998 write 0xb726ff0 thru 0xb72c568 (0x5579 bytes) 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:4999 punch from 0x7732d9f to 0x773732e, (0x458f bytes) 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:5002 trunc from 0xb72c569 to 0x25d3d2b 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:5004 writesame 0x3c37aaa thru 0x3c37f23 data_size 0x3(0x47a bytes) 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:5009 write 0x26d66f1 thru 0x26ddeb6 (0x77c6 bytes) 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:5010 write 0xc360ca thru 0xc454b6 (0xf3ed bytes) 2024-09-16T13:00:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:5011 punch from 0x3832030 to 0x383f58e, (0xd55e bytes) 2024-09-16T13:00:38.277 INFO:teuthology.orchestra.run.smithi042.stdout:5013 write 0x117b3d8 thru 0x117da66 (0x268f bytes) 2024-09-16T13:00:38.277 INFO:teuthology.orchestra.run.smithi042.stdout:5014 writesame 0xcaadff3 thru 0xcab6e64 data_size 0x1(0x8e72 bytes) 2024-09-16T13:00:38.277 INFO:teuthology.orchestra.run.smithi042.stdout:5015 read 0x2aa6d92 thru 0x2ab4785 (0xd9f4 bytes) 2024-09-16T13:00:38.277 INFO:teuthology.orchestra.run.smithi042.stdout:5016 read 0x2af944b thru 0x2b04a56 (0xb60c bytes) 2024-09-16T13:00:38.277 INFO:teuthology.orchestra.run.smithi042.stdout:5018 write 0x843bbdc thru 0x8443782 (0x7ba7 bytes) 2024-09-16T13:00:38.277 INFO:teuthology.orchestra.run.smithi042.stdout:5019 punch from 0x1a24767 to 0x1a2f32a, (0xabc3 bytes) 2024-09-16T13:00:38.277 INFO:teuthology.orchestra.run.smithi042.stdout:5020 write 0x46d2d79 thru 0x46d58eb (0x2b73 bytes) 2024-09-16T13:00:38.277 INFO:teuthology.orchestra.run.smithi042.stdout:5022 writesame 0x3a28c95 thru 0x3a2e1b0 data_size 0x1a(0x551c bytes) 2024-09-16T13:00:38.277 INFO:teuthology.orchestra.run.smithi042.stdout:5023 clone 32 order 18 su 1024 sc 5 2024-09-16T13:00:38.372 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:38.364+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:38.381 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T13:00:38.384 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:38.384 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:38.394 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:38.472 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:38.464+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:38.493 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 71.124451539000006, 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:38.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T13:00:38.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.498 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:18031 4.4 4:2649aabf:::rbd_data.11b51a3eb68b.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~5304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.328128305999996, 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94984657100000003, 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 18031 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160519+0000", 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3029999999999998e-06 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160536+0000", 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7181999999999999e-05 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160564+0000", 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971e-05 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160576+0000", 2024-09-16T13:00:38.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160669+0000", 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3282999999999994e-05 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160680+0000", 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107188+0000", 2024-09-16T13:00:38.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94650794999999999 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107644+0000", 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00045604499999999997 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.108257+0000", 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061324700000000003 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.109811+0000", 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015538749999999999 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110175+0000", 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036419399999999998 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110285+0000", 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010938900000000001 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110364+0000", 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8689000000000005e-05 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:64609 4.e 4:7394d9c8:::rbd_object_map.11b5c2ee3542:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.226919142, 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.020058463, 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.505 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 64609 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261729+0000", 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4430000000000002e-06 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261741+0000", 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261780+0000", 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8722e-05 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261798+0000", 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8150000000000001e-05 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261927+0000", 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000129243 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261948+0000", 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1047e-05 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273712+0000", 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011763704999999999 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273720+0000", 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4309999999999994e-06 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280059+0000", 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.006338744 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280297+0000", 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000237667 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281645+0000", 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001348555 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281702+0000", 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6342999999999999e-05 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281785+0000", 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2782999999999997e-05 2024-09-16T13:00:38.511 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236217 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2353152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.636860+0000", 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.85178525500000002, 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044671440999999999, 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.512 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236217 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.636860+0000", 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.636860+0000", 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.636860+0000", 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.636862+0000", 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.636869+0000", 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9380000000000003e-06 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.636873+0000", 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3579999999999999e-06 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.681261+0000", 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044388419999999998 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.681378+0000", 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117415 2024-09-16T13:00:38.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.681531+0000", 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015299199999999999 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236331 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2072576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.641813+0000", 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.84683244000000002, 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044633510000000001, 2024-09-16T13:00:38.516 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236331 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.641813+0000", 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.641813+0000", 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.641941+0000", 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012821499999999999 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.641945+0000", 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6729999999999998e-06 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.641952+0000", 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.498e-06 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.641957+0000", 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.809e-06 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.686227+0000", 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044269608000000002 2024-09-16T13:00:38.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.686313+0000", 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6572999999999999e-05 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.686446+0000", 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133134 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236332 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2091008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.641962+0000", 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.84668315599999999, 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.04491092, 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236332 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.641962+0000", 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.641962+0000", 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642092+0000", 2024-09-16T13:00:38.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013005800000000001 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642096+0000", 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4759999999999998e-06 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642105+0000", 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4029999999999996e-06 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642110+0000", 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9409999999999998e-06 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.686493+0000", 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044382554999999997 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.686684+0000", 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000191559 2024-09-16T13:00:38.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.686873+0000", 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018892799999999999 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236333 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2097152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.642116+0000", 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.84652951799999998, 2024-09-16T13:00:38.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045030146, 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236333 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642116+0000", 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642116+0000", 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642116+0000", 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4000000000000001e-07 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642119+0000", 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2680000000000001e-06 2024-09-16T13:00:38.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642125+0000", 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.066e-06 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642128+0000", 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4230000000000001e-06 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.686919+0000", 2024-09-16T13:00:38.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044790484999999998 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.687002+0000", 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3054999999999997e-05 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.687146+0000", 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014420899999999999 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236334 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2115584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.642132+0000", 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.84651314600000005, 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045267808, 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236334 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642132+0000", 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.642132+0000", 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.645509+0000", 2024-09-16T13:00:38.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003376703 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.645513+0000", 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5009999999999998e-06 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.645523+0000", 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4220000000000006e-06 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.645528+0000", 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.687197+0000", 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041668642999999998 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.687264+0000", 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6903000000000006e-05 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.687400+0000", 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136274 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236470 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2258944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.652378+0000", 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.83626739299999997, 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044400913, 2024-09-16T13:00:38.534 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.535 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.535 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236470 2024-09-16T13:00:38.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.535 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652378+0000", 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652378+0000", 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652878+0000", 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049975599999999996 2024-09-16T13:00:38.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652881+0000", 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4199999999999999e-06 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652889+0000", 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.439e-06 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652896+0000", 2024-09-16T13:00:38.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5679999999999996e-06 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.696565+0000", 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043668474999999998 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.696651+0000", 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6808e-05 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.696779+0000", 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012744699999999999 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236473 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2320384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.652939+0000", 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.83570614300000001, 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044584786000000001, 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236473 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652939+0000", 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652939+0000", 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652940+0000", 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652941+0000", 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.858e-06 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652947+0000", 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.451e-06 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652950+0000", 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.697305+0000", 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044354704000000002 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.697387+0000", 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2146999999999998e-05 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.697524+0000", 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013664100000000001 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.543 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236474 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2338816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.652954+0000", 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.83569106900000001, 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044769779000000003, 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236474 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652954+0000", 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652954+0000", 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652955+0000", 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6300000000000005e-07 2024-09-16T13:00:38.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652957+0000", 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0320000000000002e-06 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652962+0000", 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.361e-06 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652966+0000", 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5880000000000002e-06 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.697562+0000", 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044595856000000003 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.697625+0000", 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3334e-05 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.697724+0000", 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8944999999999995e-05 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236475 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2357248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.652971+0000", 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.83567472700000001, 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045038368000000002, 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.548 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236475 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652971+0000", 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.652971+0000", 2024-09-16T13:00:38.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.653070+0000", 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9840999999999995e-05 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.653074+0000", 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.366e-06 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.653082+0000", 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9670000000000001e-06 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.653087+0000", 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1229999999999999e-06 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.697754+0000", 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044666806000000003 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.697875+0000", 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121291 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.698009+0000", 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133974 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236604 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.661768+0000", 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.826877219, 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044526616999999998, 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236604 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661768+0000", 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661768+0000", 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661768+0000", 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9200000000000002e-07 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661770+0000", 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.716e-06 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661774+0000", 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0300000000000004e-06 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661777+0000", 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.791e-06 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.705952+0000", 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044174818999999997 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.706103+0000", 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015125699999999999 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.706295+0000", 2024-09-16T13:00:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019161199999999999 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236605 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2930688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.661781+0000", 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.82686464000000004, 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044705588999999997, 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236605 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661781+0000", 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661781+0000", 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661781+0000", 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6600000000000002e-07 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661783+0000", 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5850000000000001e-06 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661787+0000", 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1690000000000002e-06 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.661790+0000", 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8590000000000001e-06 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.706323+0000", 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044533199000000002 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.706389+0000", 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6614999999999998e-05 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.706486+0000", 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6796e-05 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236728 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2834432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.669030+0000", 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.81961510400000004, 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044411537000000001, 2024-09-16T13:00:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236728 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669030+0000", 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669030+0000", 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669031+0000", 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3500000000000002e-07 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669033+0000", 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2419999999999999e-06 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669040+0000", 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1119999999999998e-06 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669045+0000", 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7210000000000002e-06 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713122+0000", 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044077312 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713231+0000", 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108962 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713442+0000", 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021075299999999999 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236729 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2852864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.669049+0000", 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.819595984, 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044577390000000001, 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236729 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669049+0000", 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669049+0000", 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669050+0000", 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7e-07 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669052+0000", 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2539999999999999e-06 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669059+0000", 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5989999999999998e-06 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669062+0000", 2024-09-16T13:00:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8659999999999999e-06 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713477+0000", 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044414464000000001 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713535+0000", 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8174999999999998e-05 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713627+0000", 2024-09-16T13:00:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1562000000000002e-05 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236730 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2871296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.669067+0000", 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.81957839099999996, 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.04473071, 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236730 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669067+0000", 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669067+0000", 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669067+0000", 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2800000000000002e-07 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669069+0000", 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0999999999999998e-06 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669076+0000", 2024-09-16T13:00:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5749999999999997e-06 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669080+0000", 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713641+0000", 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044561390999999999 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713692+0000", 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0834999999999997e-05 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713798+0000", 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105314 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236731 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2895872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.669085+0000", 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.81956032400000001, 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044889597000000003, 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236731 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669085+0000", 2024-09-16T13:00:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669085+0000", 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669173+0000", 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8088000000000001e-05 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669180+0000", 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.674e-06 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669186+0000", 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8830000000000003e-06 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669191+0000", 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4890000000000003e-06 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713819+0000", 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044627627000000003 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713879+0000", 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9877999999999998e-05 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713975+0000", 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5958000000000005e-05 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236732 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2914304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.669207+0000", 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.81943839799999996, 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044912753, 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236732 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669207+0000", 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669207+0000", 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669207+0000", 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0600000000000001e-07 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669209+0000", 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9980000000000002e-06 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669213+0000", 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1999999999999996e-06 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669217+0000", 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7560000000000001e-06 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.713990+0000", 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044772495000000002 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.714040+0000", 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9792e-05 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.714120+0000", 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0105999999999995e-05 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:236733 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:22 [read 2932736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:37.669220+0000", 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.819425505, 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045171643999999997, 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 236733 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669220+0000", 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669220+0000", 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669320+0000", 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100342 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669323+0000", 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3900000000000002e-06 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669331+0000", 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3649999999999998e-06 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.669335+0000", 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3760000000000001e-06 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.714135+0000", 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044799922999999998 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.714197+0000", 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1694999999999998e-05 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:37.714391+0000", 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019455300000000001 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.588 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:38.589 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:38.589 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:38.581+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:38.689 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:38.681+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:38.697 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:38.697 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:38.706 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:38.791 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:38.784+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:38.792 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-16T13:00:38.811 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:38.811 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:38.820 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:38.831 INFO:teuthology.orchestra.run.smithi042.stdout:5024 read 0x4bc95b3 thru 0x4bca2f1 (0xd3f bytes) 2024-09-16T13:00:38.833 INFO:teuthology.orchestra.run.smithi042.stdout:5025 clone 33 order 20 su 32768 sc 5 2024-09-16T13:00:38.891 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:38.888+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:38.919 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 113.36487485000001, 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:00:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:00:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:00:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:00:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:00:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:00:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 112.539757803, 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:00:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:00:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 112.539710547, 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:00:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:00:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:00:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:00:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:00:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 112.539507611, 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:00:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 112.539476349, 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:00:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:00:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 112.539440587, 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:00:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:00:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:00:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 112.53924595700001, 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:00:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:00:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 112.53919911200001, 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:00:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.994922941, 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:00:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.994869034, 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:00:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.994826729, 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:00:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.994668304, 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:00:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.994618823, 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:00:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:00:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:00:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.994488253, 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:00:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:00:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.99430674, 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:00:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:00:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.183779073, 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:00:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.183630119, 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:00:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:00:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64507 4.1e e83/31 4:7f7c7834:::rbd_data.11b5c2ee3542.0000000000000008:head v 83'225, mlcod=81'223)", 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.144216832000001, 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021284401000000001, 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770380+0000", 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8999999999999998e-07 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777141+0000", 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067617670000000001 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777157+0000", 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777167+0000", 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777267+0000", 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100364 2024-09-16T13:00:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777304+0000", 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7165000000000001e-05 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791601+0000", 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014297275999999999 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791664+0000", 2024-09-16T13:00:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2217000000000004e-05 2024-09-16T13:00:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64509 4.15 e83/31 4:ad2e18f4:::rbd_data.11b5c2ee3542.0000000000000009:head v 83'183, mlcod=81'180)", 2024-09-16T13:00:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.137458930000001, 2024-09-16T13:00:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014054395000000001, 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777138+0000", 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778534+0000", 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013956769999999999 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778555+0000", 2024-09-16T13:00:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1046000000000001e-05 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778564+0000", 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9290000000000004e-06 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778633+0000", 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9363000000000007e-05 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778666+0000", 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135000000000003e-05 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791145+0000", 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012479743 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791192+0000", 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6156000000000003e-05 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:217754 4.17 e92/31 4:ee4048fa:::rbd_data.11b58362ce83.0000000000000002:head v 92'697, mlcod=91'693)", 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.415288876, 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011416822, 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499309+0000", 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6929999999999999e-06 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499322+0000", 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499354+0000", 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135999999999999e-05 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499365+0000", 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111e-05 2024-09-16T13:00:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499504+0000", 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013877599999999999 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499565+0000", 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1939000000000004e-05 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510591+0000", 2024-09-16T13:00:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011025874999999999 2024-09-16T13:00:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510724+0000", 2024-09-16T13:00:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013266299999999999 2024-09-16T13:00:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:39.037 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:39.037 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:39.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:39.039 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:39.032+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:39.139 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:39.132+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:39.150 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:39.150 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:39.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:39.240 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:39.232+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:39.260 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:39.260 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:39.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:39.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:39.382 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:39.376+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:39.387 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690276738000001, 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:00:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690261712000002, 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:00:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:00:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:00:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690245690000001, 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:00:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:00:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:00:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:00:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:00:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690229594000002, 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:00:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690213704000001, 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:00:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:00:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:00:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:00:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690198034000002, 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690182286999999, 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:00:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:00:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690166499, 2024-09-16T13:00:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:00:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:00:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:00:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690151177000001, 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690131252, 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:00:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:00:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:00:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:00:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:00:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690115303999999, 2024-09-16T13:00:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:00:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:00:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:00:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:00:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:00:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:00:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:00:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690100142999999, 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:00:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690084495000001, 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:00:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690069397999999, 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:00:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690054136000001, 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:00:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:00:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690038390000002, 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:00:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:00:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690022793000001, 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:00:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.690007488999999, 2024-09-16T13:00:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:00:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:00:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:00:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:00:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:00:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.689992051000001, 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:00:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:00:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.689976338000001, 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:00:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:39.483 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:39.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:39.485 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:39.486 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone32 from 0xcab6e65 (overlap 0xcab6e65) to 0x3cd0784 2024-09-16T13:00:39.486 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:39.477+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:39.498 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:39.560 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:39.589 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:39.580+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:39.605 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:39.634 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:39.651 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:39.660 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:39.705 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:39.705 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:39.696+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:39.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:39.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:39.706 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:39.706 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:39.706 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:39.706 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:39.706 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:39.706 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:39.708 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:39.708 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:39.787 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:39.781+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:39.805 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:39.805 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:39.819 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:39.888 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:39.880+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:39.972 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:39.972 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:39.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:39.989 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:39.980+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:40.089 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:40.084+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:40.119 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:40.119 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:40.132 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:40.190 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:40.184+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:40.233 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:40.233 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:40.243 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:40.280 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-16T13:00:40.291 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:40.284+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:40.350 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:40.350 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:40.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:40.394 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T13:00:40.409 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:40.400+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:40.489 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:40.489 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:40.493 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:40.484+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:40.500 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:40.594 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:40.589+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:40.597 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:40.598 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:40.607 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:40.694 INFO:teuthology.orchestra.run.smithi042.stdout:5026 read 0x19cf3fb thru 0x19d8161 (0x8d67 bytes) 2024-09-16T13:00:40.695 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:40.688+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:40.697 INFO:teuthology.orchestra.run.smithi042.stdout:5028 read 0xc5e6b17 thru 0xc5f25d4 (0xbabe bytes) 2024-09-16T13:00:40.698 INFO:teuthology.orchestra.run.smithi042.stdout:5029 read 0x7e0d115 thru 0x7e0d124 (0x10 bytes) 2024-09-16T13:00:40.698 INFO:teuthology.orchestra.run.smithi042.stdout:5030 writesame 0xa2efad8 thru 0xa2fc955 data_size 0x1(0xce7e bytes) 2024-09-16T13:00:40.704 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:40.704 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:40.714 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:40.749 INFO:teuthology.orchestra.run.smithi042.stdout:5032 trunc from 0xcab6e65 to 0xb780a95 2024-09-16T13:00:40.796 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:40.789+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:40.799 INFO:teuthology.orchestra.run.smithi042.stdout:5033 writesame 0xb8cfadd thru 0xb8d64bb data_size 0x1(0x69df bytes) 2024-09-16T13:00:40.802 INFO:teuthology.orchestra.run.smithi042.stdout:5034 write 0x5e4f053 thru 0x5e5563c (0x65ea bytes) 2024-09-16T13:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout:5038 read 0x42a00b8 thru 0x42a3d0a (0x3c53 bytes) 2024-09-16T13:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout:5041 writesame 0x48c7796 thru 0x48c7f1d data_size 0x8(0x788 bytes) 2024-09-16T13:00:40.822 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:40.822 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:40.833 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout:5042 write 0x8231b9e thru 0x823965c (0x7abf bytes) 2024-09-16T13:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout:5043 read 0x6b26040 thru 0x6b27e7b (0x1e3c bytes) 2024-09-16T13:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout:5046 punch from 0x8bae5dc to 0x8bb783a, (0x925e bytes) 2024-09-16T13:00:40.896 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:40.889+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:40.897 INFO:teuthology.orchestra.run.smithi042.stdout:5047 trunc from 0xb8d64bc to 0x6514b07 2024-09-16T13:00:40.937 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:40.937 INFO:teuthology.orchestra.run.smithi042.stdout:5050 write 0x1654860 thru 0x165c67d (0x7e1e bytes) 2024-09-16T13:00:40.948 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:40.948 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:40.957 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:40.959 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:40.959 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout:5053 writesame 0x7672b37 thru 0x7678c17 data_size 0x7(0x60e1 bytes) 2024-09-16T13:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout:5055 read 0x4bd0bc2 thru 0x4bda603 (0x9a42 bytes) 2024-09-16T13:00:40.974 INFO:teuthology.orchestra.run.smithi042.stdout:5056 trunc from 0x7678c18 to 0x5ce525b 2024-09-16T13:00:40.996 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:40.988+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:41.035 INFO:teuthology.orchestra.run.smithi042.stdout:5057 trunc from 0x5ce525b to 0x287836 2024-09-16T13:00:41.098 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:41.092+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:41.116 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:41.116 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:41.127 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:41.198 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:41.192+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:41.222 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:41.222 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:41.231 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:41.245 INFO:teuthology.orchestra.run.smithi042.stdout:5062 punch from 0x2864df to 0x287241, (0xd62 bytes) 2024-09-16T13:00:41.245 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.245 INFO:teuthology.orchestra.run.smithi042.stdout:5064 read 0xeb25c thru 0xeccb3 (0x1a58 bytes) 2024-09-16T13:00:41.247 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.247 INFO:teuthology.orchestra.run.smithi042.stdout:5066 read 0x11ebb7 thru 0x121127 (0x2571 bytes) 2024-09-16T13:00:41.249 INFO:teuthology.orchestra.run.smithi042.stdout:5068 read 0x1e81ad thru 0x1ec5d0 (0x4424 bytes) 2024-09-16T13:00:41.275 INFO:teuthology.orchestra.run.smithi042.stdout:5069 trunc from 0x287836 to 0xd6ab407 2024-09-16T13:00:41.286 INFO:teuthology.orchestra.run.smithi042.stdout:5070 punch from 0xc97c7fd to 0xc987165, (0xa968 bytes) 2024-09-16T13:00:41.299 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:41.292+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:41.301 INFO:teuthology.orchestra.run.smithi042.stdout:5072 write 0xe6c1853 thru 0xe6cbaad (0xa25b bytes) 2024-09-16T13:00:41.311 INFO:teuthology.orchestra.run.smithi042.stdout:5074 write 0x442a7d5 thru 0x4435b02 (0xb32e bytes) 2024-09-16T13:00:41.320 INFO:teuthology.orchestra.run.smithi042.stdout:5075 read 0x33d7a0e thru 0x33e1995 (0x9f88 bytes) 2024-09-16T13:00:41.320 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.320 INFO:teuthology.orchestra.run.smithi042.stdout:5078 writesame 0x4427226 thru 0x4430863 data_size 0x1(0x963e bytes) 2024-09-16T13:00:41.327 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:41.327 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:41.338 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:41.363 INFO:teuthology.orchestra.run.smithi042.stdout:5080 trunc from 0xe6cbaae to 0x420d5fa 2024-09-16T13:00:41.400 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:41.393+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T13:00:41.427 INFO:teuthology.orchestra.run.smithi042.stdout:5083 write 0xe55e937 thru 0xe5683ad (0x9a77 bytes) 2024-09-16T13:00:41.431 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:41.431 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:41.436 INFO:teuthology.orchestra.run.smithi042.stdout:5084 punch from 0xb392e6b to 0xb3a07db, (0xd970 bytes) 2024-09-16T13:00:41.436 INFO:teuthology.orchestra.run.smithi042.stdout:5085 write 0x1921c6a thru 0x1930859 (0xebf0 bytes) 2024-09-16T13:00:41.442 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:41.446 INFO:teuthology.orchestra.run.smithi042.stdout:5086 read 0x7e4379f thru 0x7e4c2f8 (0x8b5a bytes) 2024-09-16T13:00:41.446 INFO:teuthology.orchestra.run.smithi042.stdout:5087 write 0xf8f8cf thru 0xf9d18a (0xd8bc bytes) 2024-09-16T13:00:41.458 INFO:teuthology.orchestra.run.smithi042.stdout:5088 read 0x2b9dfb2 thru 0x2bacd6a (0xedb9 bytes) 2024-09-16T13:00:41.458 INFO:teuthology.orchestra.run.smithi042.stdout:5089 write 0xd02d032 thru 0xd033ec4 (0x6e93 bytes) 2024-09-16T13:00:41.465 INFO:teuthology.orchestra.run.smithi042.stdout:5090 write 0xd8f08e0 thru 0xd8f5ed2 (0x55f3 bytes) 2024-09-16T13:00:41.470 INFO:teuthology.orchestra.run.smithi042.stdout:5091 write 0x2e9d647 thru 0x2e9e94f (0x1309 bytes) 2024-09-16T13:00:41.477 INFO:teuthology.orchestra.run.smithi042.stdout:5093 write 0xdc84513 thru 0xdc8a1ba (0x5ca8 bytes) 2024-09-16T13:00:41.501 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:41.492+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout:5094 trunc from 0xe5683ae to 0x8371267 2024-09-16T13:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout:5095 writesame 0xc80ddae thru 0xc817a9e data_size 0x1(0x9cf1 bytes) 2024-09-16T13:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout:5097 write 0x3f2bb3e thru 0x3f35639 (0x9afc bytes) 2024-09-16T13:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout:5098 trunc from 0xc817a9f to 0xb6f951a 2024-09-16T13:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:5099 punch from 0xaf85463 to 0xaf8a08f, (0x4c2c bytes) 2024-09-16T13:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout:5100 trunc from 0xb6f951a to 0x96918a0 2024-09-16T13:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout:5101 read 0x5095e91 thru 0x509985e (0x39ce bytes) 2024-09-16T13:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout:5102 write 0x494bbda thru 0x495b65f (0xfa86 bytes) 2024-09-16T13:00:41.587 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:41.587 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:41.596 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:41.598 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.599 INFO:teuthology.orchestra.run.smithi042.stdout:5104 punch from 0x50342ba to 0x503921d, (0x4f63 bytes) 2024-09-16T13:00:41.599 INFO:teuthology.orchestra.run.smithi042.stdout:5105 punch from 0x5c723f4 to 0x5c7f84d, (0xd459 bytes) 2024-09-16T13:00:41.599 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.599 INFO:teuthology.orchestra.run.smithi042.stdout:5107 punch from 0xf929f1 to 0xf98c7a, (0x6289 bytes) 2024-09-16T13:00:41.599 INFO:teuthology.orchestra.run.smithi042.stdout:5108 read 0x4d89e3e thru 0x4d936ba (0x987d bytes) 2024-09-16T13:00:41.599 INFO:teuthology.orchestra.run.smithi042.stdout:5109 writesame 0x5b467cf thru 0x5b4cfc2 data_size 0x2(0x67f4 bytes) 2024-09-16T13:00:41.601 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:41.593+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:41.612 INFO:teuthology.orchestra.run.smithi042.stdout:5110 read 0x333379d thru 0x333c736 (0x8f9a bytes) 2024-09-16T13:00:41.613 INFO:teuthology.orchestra.run.smithi042.stdout:5111 write 0x178f6fb thru 0x1791071 (0x1977 bytes) 2024-09-16T13:00:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.637 INFO:teuthology.orchestra.run.smithi042.stdout:5114 writesame 0x9af57ad thru 0x9afc590 data_size 0xd(0x6de4 bytes) 2024-09-16T13:00:41.637 INFO:teuthology.orchestra.run.smithi042.stdout:5115 read 0x71be1da thru 0x71c0175 (0x1f9c bytes) 2024-09-16T13:00:41.637 INFO:teuthology.orchestra.run.smithi042.stdout:5116 read 0x3b35ae2 thru 0x3b3af60 (0x547f bytes) 2024-09-16T13:00:41.637 INFO:teuthology.orchestra.run.smithi042.stdout:5119 read 0x6ccc0ff thru 0x6cd882e (0xc730 bytes) 2024-09-16T13:00:41.637 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.637 INFO:teuthology.orchestra.run.smithi042.stdout:5121 read 0x61bfe41 thru 0x61c95f9 (0x97b9 bytes) 2024-09-16T13:00:41.637 INFO:teuthology.orchestra.run.smithi042.stdout:5122 read 0x3e7327e thru 0x3e77476 (0x41f9 bytes) 2024-09-16T13:00:41.658 INFO:teuthology.orchestra.run.smithi042.stdout:5123 write 0xcd2994c thru 0xcd3051a (0x6bcf bytes) 2024-09-16T13:00:41.667 INFO:teuthology.orchestra.run.smithi042.stdout:5125 read 0x1e365f6 thru 0x1e3a096 (0x3aa1 bytes) 2024-09-16T13:00:41.667 INFO:teuthology.orchestra.run.smithi042.stdout:5128 write 0x630c467 thru 0x6315a10 (0x95aa bytes) 2024-09-16T13:00:41.678 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.678 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xee59a0c 2024-09-16T13:00:41.684 INFO:teuthology.orchestra.run.smithi042.stdout:5132 trunc from 0xcd3051b to 0xee59a0c 2024-09-16T13:00:41.696 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:41.696 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:41.697 INFO:teuthology.orchestra.run.smithi042.stdout:5133 write 0x4046258 thru 0x404e876 (0x861f bytes) 2024-09-16T13:00:41.701 INFO:teuthology.orchestra.run.smithi042.stdout:5134 writesame 0xc33d247 thru 0xc34a805 data_size 0x1(0xd5bf bytes) 2024-09-16T13:00:41.702 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:41.696+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:41.705 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:41.709 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:41.737 INFO:teuthology.orchestra.run.smithi042.stdout:5135 read 0x37d1633 thru 0x37e0cb0 (0xf67e bytes) 2024-09-16T13:00:41.738 INFO:teuthology.orchestra.run.smithi042.stdout:5136 flatten 2024-09-16T13:00:41.748 INFO:teuthology.orchestra.run.smithi042.stdout:5137 read 0xa4b64dd thru 0xa4b97fb (0x331f bytes) 2024-09-16T13:00:41.763 INFO:teuthology.orchestra.run.smithi042.stdout:5138 trunc from 0xee59a0c to 0x6fb7045 2024-09-16T13:00:41.785 INFO:teuthology.orchestra.run.smithi042.stdout:5139 punch from 0x645fe9b to 0x6460f34, (0x1099 bytes) 2024-09-16T13:00:41.785 INFO:teuthology.orchestra.run.smithi042.stdout:5141 punch from 0x4998392 to 0x49a44aa, (0xc118 bytes) 2024-09-16T13:00:41.802 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:41.797+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:41.803 INFO:teuthology.orchestra.run.smithi042.stdout:5142 write 0xa937afb thru 0xa942e77 (0xb37d bytes) 2024-09-16T13:00:41.806 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:41.806 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:41.814 INFO:teuthology.orchestra.run.smithi042.stdout:5143 read 0x59cc0a0 thru 0x59d3768 (0x76c9 bytes) 2024-09-16T13:00:41.814 INFO:teuthology.orchestra.run.smithi042.stdout:5144 read 0x412f060 thru 0x41387c6 (0x9767 bytes) 2024-09-16T13:00:41.815 INFO:teuthology.orchestra.run.smithi042.stdout:5145 read 0x4a967f1 thru 0x4aa0874 (0xa084 bytes) 2024-09-16T13:00:41.816 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:41.817 INFO:teuthology.orchestra.run.smithi042.stdout:5146 punch from 0x3704e4e to 0x37052c8, (0x47a bytes) 2024-09-16T13:00:41.823 INFO:teuthology.orchestra.run.smithi042.stdout:5147 trunc from 0xa942e78 to 0xe3d4dce 2024-09-16T13:00:41.832 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.832 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.832 INFO:teuthology.orchestra.run.smithi042.stdout:5150 punch from 0x3a67c74 to 0x3a6d9ea, (0x5d76 bytes) 2024-09-16T13:00:41.833 INFO:teuthology.orchestra.run.smithi042.stdout:5152 write 0xf56b40 thru 0xf62396 (0xb857 bytes) 2024-09-16T13:00:41.840 INFO:teuthology.orchestra.run.smithi042.stdout:5153 read 0xad222a2 thru 0xad29cae (0x7a0d bytes) 2024-09-16T13:00:41.852 INFO:teuthology.orchestra.run.smithi042.stdout:5155 trunc from 0xe3d4dce to 0x700ce01 2024-09-16T13:00:41.870 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:41.876 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.876 INFO:teuthology.orchestra.run.smithi042.stdout:5161 write 0x30e069d thru 0x30e8100 (0x7a64 bytes) 2024-09-16T13:00:41.886 INFO:teuthology.orchestra.run.smithi042.stdout:5162 writesame 0xfe901b thru 0xff0430 data_size 0x1(0x7416 bytes) 2024-09-16T13:00:41.896 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:41.903 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:41.897+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:41.907 INFO:teuthology.orchestra.run.smithi042.stdout:5163 punch from 0x5cda888 to 0x5cda90c, (0x84 bytes) 2024-09-16T13:00:41.907 INFO:teuthology.orchestra.run.smithi042.stdout:5165 read 0x50319a2 thru 0x504109e (0xf6fd bytes) 2024-09-16T13:00:41.914 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:41.924 INFO:teuthology.orchestra.run.smithi042.stdout:5166 write 0xb32d411 thru 0xb3364ab (0x909b bytes) 2024-09-16T13:00:41.933 INFO:teuthology.orchestra.run.smithi042.stdout:5167 read 0x33d5a01 thru 0x33debbc (0x91bc bytes) 2024-09-16T13:00:41.933 INFO:teuthology.orchestra.run.smithi042.stdout:5169 writesame 0x4333f55 thru 0x433d188 data_size 0x6(0x9234 bytes) 2024-09-16T13:00:41.935 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:41.935 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:41.935 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:41.935 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:41.938 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:41.939 INFO:teuthology.orchestra.run.smithi042.stdout:5170 write 0xa65b870 thru 0xa66888c (0xd01d bytes) 2024-09-16T13:00:41.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:41.951 INFO:teuthology.orchestra.run.smithi042.stdout:5171 read 0xb2309b thru 0xb2cc11 (0x9b77 bytes) 2024-09-16T13:00:41.951 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:41.951 INFO:teuthology.orchestra.run.smithi042.stdout:5173 write 0xa98e68d thru 0xa98ebaf (0x523 bytes) 2024-09-16T13:00:41.953 INFO:teuthology.orchestra.run.smithi042.stdout:5174 write 0x25525a4 thru 0x255ede5 (0xc842 bytes) 2024-09-16T13:00:41.962 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:41.977 INFO:teuthology.orchestra.run.smithi042.stdout:5176 trunc from 0xb3364ac to 0x178fb25 2024-09-16T13:00:41.997 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:42.003 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:41.996+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:42.008 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:42.021 INFO:teuthology.orchestra.run.smithi042.stdout:5179 write 0x40de5a8 thru 0x40eddc7 (0xf820 bytes) 2024-09-16T13:00:42.031 INFO:teuthology.orchestra.run.smithi042.stdout:5181 read 0x3a0fc12 thru 0x3a1a600 (0xa9ef bytes) 2024-09-16T13:00:42.031 INFO:teuthology.orchestra.run.smithi042.stdout:5184 punch from 0x2319c27 to 0x2329852, (0xfc2b bytes) 2024-09-16T13:00:42.031 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:42.031 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:42.031 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:42.031 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:42.032 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:42.032 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:42.032 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:42.032 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:42.032 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:42.035 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:42.035 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:42.046 INFO:teuthology.orchestra.run.smithi042.stdout:5185 write 0x7820afc thru 0x782288b (0x1d90 bytes) 2024-09-16T13:00:42.054 INFO:teuthology.orchestra.run.smithi042.stdout:5186 read 0x251f283 thru 0x2523bbe (0x493c bytes) 2024-09-16T13:00:42.055 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:42.067 INFO:teuthology.orchestra.run.smithi042.stdout:5190 write 0x8649c2d thru 0x8653cc9 (0xa09d bytes) 2024-09-16T13:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout:5191 read 0x1731a57 thru 0x1736da7 (0x5351 bytes) 2024-09-16T13:00:42.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout:5192 write 0xed84da7 thru 0xed8df6e (0x91c8 bytes) 2024-09-16T13:00:42.105 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:42.097+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout:5193 read 0x4c5da8c thru 0x4c650bc (0x7631 bytes) 2024-09-16T13:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout:5194 trunc from 0xed8df6f to 0x4646f81 2024-09-16T13:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout:5195 punch from 0x34df50 to 0x352627, (0x46d7 bytes) 2024-09-16T13:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout:5196 punch from 0x43fbd06 to 0x43fe493, (0x278d bytes) 2024-09-16T13:00:42.169 INFO:teuthology.orchestra.run.smithi042.stdout:5197 write 0x5d417c5 thru 0x5d511c4 (0xfa00 bytes) 2024-09-16T13:00:42.180 INFO:teuthology.orchestra.run.smithi042.stdout:5200 read 0x1093752 thru 0x109754d (0x3dfc bytes) 2024-09-16T13:00:42.181 INFO:teuthology.orchestra.run.smithi042.stdout:5201 read 0x1207b7c thru 0x120ab92 (0x3017 bytes) 2024-09-16T13:00:42.182 INFO:teuthology.orchestra.run.smithi042.stdout:5204 read 0x497dc93 thru 0x49802d7 (0x2645 bytes) 2024-09-16T13:00:42.182 INFO:teuthology.orchestra.run.smithi042.stdout:5207 punch from 0x5be4ae2 to 0x5be6db9, (0x22d7 bytes) 2024-09-16T13:00:42.182 INFO:teuthology.orchestra.run.smithi042.stdout:5208 read 0x2109ae3 thru 0x21159db (0xbef9 bytes) 2024-09-16T13:00:42.182 INFO:teuthology.orchestra.run.smithi042.stdout:5211 write 0x4c03f1f thru 0x4c09ceb (0x5dcd bytes) 2024-09-16T13:00:42.189 INFO:teuthology.orchestra.run.smithi042.stdout:5213 write 0x9825ad thru 0x98daff (0xb553 bytes) 2024-09-16T13:00:42.206 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:42.197+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.511046571000001, 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:00:42.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:00:42.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:00:42.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:00:42.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:00:42.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:00:42.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.214 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.511031545000002, 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:00:42.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.511015523000001, 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.219 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510999427000002, 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:00:42.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:00:42.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510983537000001, 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510967867000002, 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:00:42.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:00:42.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510952119999999, 2024-09-16T13:00:42.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:00:42.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:00:42.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510936332, 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:00:42.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.245 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510921010000001, 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:00:42.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:00:42.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:00:42.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:00:42.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:00:42.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510901085, 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:00:42.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.254 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510885136999999, 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:00:42.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:00:42.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510869975999999, 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:42.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:00:42.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.263 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510854328000001, 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:00:42.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510839230999999, 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:00:42.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:00:42.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510823969, 2024-09-16T13:00:42.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510808223000002, 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:42.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:00:42.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:00:42.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510792626000001, 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510777321999999, 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:00:42.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510761884000001, 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.510746171000001, 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:00:42.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:00:42.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:00:42.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.298 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.298 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.298 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:42.299 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:42.301 INFO:teuthology.orchestra.run.smithi042.stdout:5216 write 0xda449f8 thru 0xda4dd27 (0x9330 bytes) 2024-09-16T13:00:42.301 INFO:teuthology.orchestra.run.smithi042.stdout:5217 writesame 0x4e26d56 thru 0x4e30580 data_size 0x35(0x982b bytes) 2024-09-16T13:00:42.301 INFO:teuthology.orchestra.run.smithi042.stdout:5218 read 0x8e0a27a thru 0x8e1689a (0xc621 bytes) 2024-09-16T13:00:42.301 INFO:teuthology.orchestra.run.smithi042.stdout:5219 read 0x1f1f1cc thru 0x1f24f6e (0x5da3 bytes) 2024-09-16T13:00:42.301 INFO:teuthology.orchestra.run.smithi042.stdout:5220 read 0xa7de05a thru 0xa7e1b65 (0x3b0c bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5223 write 0xd85fdbb thru 0xd86f3d8 (0xf61e bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5224 read 0xc7a435 thru 0xc89f01 (0xfacd bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5225 write 0xd59403f thru 0xd595277 (0x1239 bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5227 punch from 0x4887629 to 0x489411a, (0xcaf1 bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5232 read 0x62e42b8 thru 0x62e62a3 (0x1fec bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5233 writesame 0xbc61d64 thru 0xbc649e5 data_size 0x12(0x2c82 bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5234 read 0xc1eba73 thru 0xc1f8df2 (0xd380 bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5235 write 0x68566a1 thru 0x685a000 (0x3960 bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5236 read 0xa8bcb5e thru 0xa8c6e10 (0xa2b3 bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5237 read 0x11b3121 thru 0x11b64d9 (0x33b9 bytes) 2024-09-16T13:00:42.302 INFO:teuthology.orchestra.run.smithi042.stdout:5238 trunc from 0xda4dd28 to 0xc6b87f6 2024-09-16T13:00:42.303 INFO:teuthology.orchestra.run.smithi042.stdout:5240 punch from 0x372e0c0 to 0x373b009, (0xcf49 bytes) 2024-09-16T13:00:42.303 INFO:teuthology.orchestra.run.smithi042.stdout:5241 write 0xbec700a thru 0xbecf4a4 (0x849b bytes) 2024-09-16T13:00:42.307 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:42.301+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:42.326 INFO:teuthology.orchestra.run.smithi042.stdout:5242 trunc from 0xc6b87f6 to 0x2f79c95 2024-09-16T13:00:42.352 INFO:teuthology.orchestra.run.smithi042.stdout:5243 read 0x54bba5 thru 0x54f8d0 (0x3d2c bytes) 2024-09-16T13:00:42.352 INFO:teuthology.orchestra.run.smithi042.stdout:5244 read 0xf946e4 thru 0xf99e9f (0x57bc bytes) 2024-09-16T13:00:42.353 INFO:teuthology.orchestra.run.smithi042.stdout:5245 read 0x2522365 thru 0x252ed05 (0xc9a1 bytes) 2024-09-16T13:00:42.357 INFO:teuthology.orchestra.run.smithi042.stdout:5250 trunc from 0x2f79c95 to 0x16c20e5 2024-09-16T13:00:42.402 INFO:teuthology.orchestra.run.smithi042.stdout:5251 writesame 0xd36e59f thru 0xd371661 data_size 0x13(0x30c3 bytes) 2024-09-16T13:00:42.403 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:42.403 INFO:teuthology.orchestra.run.smithi042.stdout:5253 write 0xc7e90a4 thru 0xc7efdf2 (0x6d4f bytes) 2024-09-16T13:00:42.408 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:42.401+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:42.409 INFO:teuthology.orchestra.run.smithi042.stdout:5254 read 0x6bee5e thru 0x6c962d (0xa7d0 bytes) 2024-09-16T13:00:42.410 INFO:teuthology.orchestra.run.smithi042.stdout:5255 write 0xc8f5629 thru 0xc8fe77d (0x9155 bytes) 2024-09-16T13:00:42.417 INFO:teuthology.orchestra.run.smithi042.stdout:5256 read 0xc43a0d5 thru 0xc43efba (0x4ee6 bytes) 2024-09-16T13:00:42.418 INFO:teuthology.orchestra.run.smithi042.stdout:5257 read 0x8c33314 thru 0x8c35562 (0x224f bytes) 2024-09-16T13:00:42.418 INFO:teuthology.orchestra.run.smithi042.stdout:5258 writesame 0x5d8f24e thru 0x5d98665 data_size 0x1c(0x9418 bytes) 2024-09-16T13:00:42.425 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-16T13:00:42.425 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T13:00:42.425 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T13:00:42.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T13:00:42.429 INFO:teuthology.orchestra.run.smithi042.stdout:5259 read 0x306b777 thru 0x30731c8 (0x7a52 bytes) 2024-09-16T13:00:42.429 INFO:teuthology.orchestra.run.smithi042.stdout:5260 writesame 0x16742ed thru 0x167ea95 data_size 0x39(0xa7a9 bytes) 2024-09-16T13:00:42.432 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:42.432 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:42.432 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:42.432 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:42.433 INFO:teuthology.orchestra.run.smithi042.stdout:5261 writesame 0x83481de thru 0x834f251 data_size 0x6(0x7074 bytes) 2024-09-16T13:00:42.442 INFO:teuthology.orchestra.run.smithi042.stdout:5262 write 0x82a041b thru 0x82a2ee9 (0x2acf bytes) 2024-09-16T13:00:42.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:42.448 INFO:teuthology.orchestra.run.smithi042.stdout:5263 write 0x3d831b thru 0x3dabf4 (0x28da bytes) 2024-09-16T13:00:42.451 INFO:teuthology.orchestra.run.smithi042.stdout:5264 punch from 0x3aa75b3 to 0x3aaae3e, (0x388b bytes) 2024-09-16T13:00:42.451 INFO:teuthology.orchestra.run.smithi042.stdout:5266 punch from 0xb5377c9 to 0xb53c830, (0x5067 bytes) 2024-09-16T13:00:42.451 INFO:teuthology.orchestra.run.smithi042.stdout:5268 punch from 0x44cfefd to 0x44d1fea, (0x20ed bytes) 2024-09-16T13:00:42.451 INFO:teuthology.orchestra.run.smithi042.stdout:5269 read 0x424f621 thru 0x425407f (0x4a5f bytes) 2024-09-16T13:00:42.451 INFO:teuthology.orchestra.run.smithi042.stdout:5270 read 0x4191c5c thru 0x41996f8 (0x7a9d bytes) 2024-09-16T13:00:42.451 INFO:teuthology.orchestra.run.smithi042.stdout:5271 read 0x978b3aa thru 0x9799f24 (0xeb7b bytes) 2024-09-16T13:00:42.451 INFO:teuthology.orchestra.run.smithi042.stdout:5272 read 0x9b1056 thru 0x9baa14 (0x99bf bytes) 2024-09-16T13:00:42.452 INFO:teuthology.orchestra.run.smithi042.stdout:5273 read 0x231924b thru 0x23262c0 (0xd076 bytes) 2024-09-16T13:00:42.452 INFO:teuthology.orchestra.run.smithi042.stdout:5274 punch from 0x2a08a27 to 0x2a17e17, (0xf3f0 bytes) 2024-09-16T13:00:42.452 INFO:teuthology.orchestra.run.smithi042.stdout:5275 write 0x97b3494 thru 0x97bc58e (0x90fb bytes) 2024-09-16T13:00:42.461 INFO:teuthology.orchestra.run.smithi042.stdout:5276 punch from 0x904f5c7 to 0x9054e01, (0x583a bytes) 2024-09-16T13:00:42.461 INFO:teuthology.orchestra.run.smithi042.stdout:5277 read 0x69eec8d thru 0x69fa837 (0xbbab bytes) 2024-09-16T13:00:42.462 INFO:teuthology.orchestra.run.smithi042.stdout:5278 write 0xb485a77 thru 0xb4868b5 (0xe3f bytes) 2024-09-16T13:00:42.466 INFO:teuthology.orchestra.run.smithi042.stdout:5279 write 0x6215feb thru 0x621cd83 (0x6d99 bytes) 2024-09-16T13:00:42.475 INFO:teuthology.orchestra.run.smithi042.stdout:5280 punch from 0x2ac39ec to 0x2ac8dda, (0x53ee bytes) 2024-09-16T13:00:42.475 INFO:teuthology.orchestra.run.smithi042.stdout:5282 read 0xc1209e9 thru 0xc120bcc (0x1e4 bytes) 2024-09-16T13:00:42.475 INFO:teuthology.orchestra.run.smithi042.stdout:5283 read 0xc5b098 thru 0xc644b5 (0x941e bytes) 2024-09-16T13:00:42.475 INFO:teuthology.orchestra.run.smithi042.stdout:5284 writesame 0x15ee3c1 thru 0x15f325b data_size 0x1(0x4e9b bytes) 2024-09-16T13:00:42.504 INFO:teuthology.orchestra.run.smithi042.stdout:5285 trunc from 0xd371662 to 0x78310bc 2024-09-16T13:00:42.509 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:42.500+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:42.537 INFO:teuthology.orchestra.run.smithi042.stdout:5288 write 0xada530b thru 0xadae52c (0x9222 bytes) 2024-09-16T13:00:42.546 INFO:teuthology.orchestra.run.smithi042.stdout:5289 punch from 0x994570c to 0x99493a7, (0x3c9b bytes) 2024-09-16T13:00:42.546 INFO:teuthology.orchestra.run.smithi042.stdout:5291 write 0x3004258 thru 0x3006cab (0x2a54 bytes) 2024-09-16T13:00:42.552 INFO:teuthology.orchestra.run.smithi042.stdout:5293 write 0x735b97 thru 0x742e60 (0xd2ca bytes) 2024-09-16T13:00:42.555 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:42.555 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:42.555 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:42.555 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:42.555 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:42.561 INFO:teuthology.orchestra.run.smithi042.stdout:5294 writesame 0x637fafd thru 0x638278d data_size 0x1(0x2c91 bytes) 2024-09-16T13:00:42.566 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:42.577 INFO:teuthology.orchestra.run.smithi042.stdout:5295 trunc from 0xadae52d to 0xe0a18f4 2024-09-16T13:00:42.587 INFO:teuthology.orchestra.run.smithi042.stdout:5296 writesame 0xb0b8114 thru 0xb0bc4d5 data_size 0x3b(0x43c2 bytes) 2024-09-16T13:00:42.594 INFO:teuthology.orchestra.run.smithi042.stdout:5297 read 0x6752bb4 thru 0x6757c94 (0x50e1 bytes) 2024-09-16T13:00:42.594 INFO:teuthology.orchestra.run.smithi042.stdout:5298 read 0x228782f thru 0x228dfed (0x67bf bytes) 2024-09-16T13:00:42.594 INFO:teuthology.orchestra.run.smithi042.stdout:5299 read 0x2e9ae82 thru 0x2ea807a (0xd1f9 bytes) 2024-09-16T13:00:42.594 INFO:teuthology.orchestra.run.smithi042.stdout:5301 read 0xc250fbf thru 0xc25b219 (0xa25b bytes) 2024-09-16T13:00:42.594 INFO:teuthology.orchestra.run.smithi042.stdout:5302 punch from 0xda0b570 to 0xda1653a, (0xafca bytes) 2024-09-16T13:00:42.595 INFO:teuthology.orchestra.run.smithi042.stdout:5303 write 0x98595b4 thru 0x985cf1a (0x3967 bytes) 2024-09-16T13:00:42.601 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:42.601 INFO:teuthology.orchestra.run.smithi042.stdout:5306 writesame 0x40adf3a thru 0x40b679f data_size 0x1(0x8866 bytes) 2024-09-16T13:00:42.610 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:42.601+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:42.639 INFO:teuthology.orchestra.run.smithi042.stdout:5307 trunc from 0xe0a18f4 to 0x593024b 2024-09-16T13:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout:5308 write 0x894f097 thru 0x8956572 (0x74dc bytes) 2024-09-16T13:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166216 4.f 4:f0f74f87:::rbd_data.11b5c2ee3542.0000000000000003:1d [read 8261632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.954225+0000", 2024-09-16T13:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.734318848999999, 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078866292000000005, 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166216 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954225+0000", 2024-09-16T13:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954225+0000", 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954225+0000", 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.75e-07 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954227+0000", 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0650000000000001e-06 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954233+0000", 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4990000000000002e-06 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954236+0000", 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3050000000000001e-06 2024-09-16T13:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.032951+0000", 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078714471999999994 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033028+0000", 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7427000000000003e-05 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033091+0000", 2024-09-16T13:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3048999999999999e-05 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166217 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8597504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.954240+0000", 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.734303629999999, 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079025498999999999, 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166217 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954240+0000", 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.954240+0000", 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955245+0000", 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010046879999999999 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955250+0000", 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0969999999999996e-06 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955260+0000", 2024-09-16T13:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0614e-05 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955266+0000", 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6910000000000002e-06 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033117+0000", 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077851358999999995 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033189+0000", 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1904999999999997e-05 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033265+0000", 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6144999999999997e-05 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166224 4.f 4:f0f74f87:::rbd_data.11b5c2ee3542.0000000000000003:1d [read 8470528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955341+0000", 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.733202984, 2024-09-16T13:00:42.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078746751000000004, 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166224 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955341+0000", 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955341+0000", 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955341+0000", 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7500000000000001e-07 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955343+0000", 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0880000000000002e-06 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955350+0000", 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5400000000000001e-06 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955353+0000", 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.591e-06 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.033952+0000", 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078598960999999995 2024-09-16T13:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034028+0000", 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5523000000000002e-05 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034087+0000", 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9673000000000003e-05 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166226 4.f 4:f0f74f87:::rbd_data.11b5c2ee3542.0000000000000003:1d [read 8515584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955357+0000", 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.733186664, 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078887578999999999, 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166226 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955357+0000", 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955357+0000", 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955357+0000", 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955359+0000", 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955365+0000", 2024-09-16T13:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7429999999999999e-06 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955368+0000", 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3400000000000002e-06 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034114+0000", 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078745599999999999 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034185+0000", 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1240000000000002e-05 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034244+0000", 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9361000000000003e-05 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166228 4.f 4:f0f74f87:::rbd_data.11b5c2ee3542.0000000000000003:1d [read 8560640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955373+0000", 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.733170880999999, 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079032832999999997, 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166228 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955373+0000", 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955373+0000", 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955373+0000", 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955375+0000", 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955380+0000", 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955384+0000", 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.625e-06 2024-09-16T13:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034269+0000", 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078884923999999995 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034344+0000", 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4692999999999998e-05 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034406+0000", 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2003999999999998e-05 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166230 4.f 4:f0f74f87:::rbd_data.11b5c2ee3542.0000000000000003:1d [read 8605696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955388+0000", 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.733155632000001, 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079187638000000005, 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166230 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955388+0000", 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955388+0000", 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955388+0000", 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0399999999999996e-07 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955390+0000", 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.979e-06 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955396+0000", 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1529999999999996e-06 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955399+0000", 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.597e-06 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034432+0000", 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079032701999999996 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034511+0000", 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8936000000000005e-05 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034576+0000", 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4766999999999998e-05 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166286 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8151040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955403+0000", 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.733140452000001, 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079434322000000002, 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166286 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955403+0000", 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955403+0000", 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955404+0000", 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8999999999999997e-07 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955406+0000", 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9039999999999999e-06 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955411+0000", 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8209999999999998e-06 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955415+0000", 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.546e-06 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034653+0000", 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079238266000000002 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034743+0000", 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0199999999999997e-05 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034837+0000", 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4094999999999999e-05 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166288 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8196096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955419+0000", 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.733124821000001, 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079603004000000005, 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166288 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955419+0000", 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955419+0000", 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955419+0000", 2024-09-16T13:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9000000000000002e-07 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955425+0000", 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0739999999999998e-06 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955432+0000", 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.754e-06 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955445+0000", 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2969999999999999e-05 2024-09-16T13:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034866+0000", 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079420705999999994 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.034938+0000", 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2479999999999997e-05 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035022+0000", 2024-09-16T13:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3629999999999997e-05 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166289 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8241152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955450+0000", 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.733093642, 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079755874000000004, 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166289 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955450+0000", 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955450+0000", 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955450+0000", 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955453+0000", 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2570000000000002e-06 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955459+0000", 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1700000000000002e-06 2024-09-16T13:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955463+0000", 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8820000000000003e-06 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035048+0000", 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079585521000000006 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035122+0000", 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3971e-05 2024-09-16T13:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035206+0000", 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3607000000000001e-05 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166290 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8286208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955467+0000", 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.733076106, 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079938254, 2024-09-16T13:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166290 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955467+0000", 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955467+0000", 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955468+0000", 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2500000000000001e-07 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955470+0000", 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9460000000000001e-06 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955475+0000", 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6500000000000001e-06 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955479+0000", 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6339999999999999e-06 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035238+0000", 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079758684999999996 2024-09-16T13:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035319+0000", 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1392999999999998e-05 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035406+0000", 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6520999999999994e-05 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166291 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8331264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955483+0000", 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.733060364, 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080127871000000003, 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166291 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955483+0000", 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955483+0000", 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955536+0000", 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2454000000000003e-05 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955539+0000", 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5420000000000001e-06 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955548+0000", 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.32e-06 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955552+0000", 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.51e-06 2024-09-16T13:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035446+0000", 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079893756999999996 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035526+0000", 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0517000000000002e-05 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035611+0000", 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4771000000000005e-05 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166292 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8376320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955557+0000", 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.732986288999999, 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080247081999999997, 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166292 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955557+0000", 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955557+0000", 2024-09-16T13:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955771+0000", 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021362900000000001 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955776+0000", 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1340000000000003e-06 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955847+0000", 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0865000000000007e-05 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955860+0000", 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2951e-05 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035644+0000", 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079784069999999999 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035721+0000", 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6763999999999999e-05 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035804+0000", 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3669000000000001e-05 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166293 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8404992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.955867+0000", 2024-09-16T13:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.732676311000001, 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080124405999999995, 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166293 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955867+0000", 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.955867+0000", 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956289+0000", 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00042214200000000003 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956294+0000", 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4519999999999999e-06 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956307+0000", 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2802999999999999e-05 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956312+0000", 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5489999999999997e-06 2024-09-16T13:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035831+0000", 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079518988999999998 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035906+0000", 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4995999999999998e-05 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.035992+0000", 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5475000000000004e-05 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166295 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8450048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.956317+0000", 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.732226683, 2024-09-16T13:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079894277, 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166295 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956317+0000", 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.956317+0000", 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957407+0000", 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010904319999999999 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957412+0000", 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1710000000000001e-06 2024-09-16T13:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957872+0000", 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045938700000000001 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957880+0000", 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5359999999999992e-06 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036026+0000", 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078145237000000006 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036116+0000", 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0354999999999998e-05 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036211+0000", 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5159000000000002e-05 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166300 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8585216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.957930+0000", 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.730613691, 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920253999999995, 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166300 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957930+0000", 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957930+0000", 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957930+0000", 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6699999999999999e-07 2024-09-16T13:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957932+0000", 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1289999999999999e-06 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957938+0000", 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0229999999999998e-06 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957942+0000", 2024-09-16T13:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8039999999999999e-06 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036665+0000", 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078722996000000003 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036756+0000", 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0742999999999995e-05 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036850+0000", 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4092000000000006e-05 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166303 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 8630272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.957947+0000", 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.730597055, 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079099346000000001, 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166303 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957947+0000", 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957947+0000", 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957947+0000", 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957949+0000", 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0889999999999998e-06 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957954+0000", 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7559999999999999e-06 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.957956+0000", 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.154e-06 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036877+0000", 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920855999999998 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.036963+0000", 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6297000000000004e-05 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.037046+0000", 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2791000000000001e-05 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166494 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 7073792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.963734+0000", 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.724809279, 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078831144000000006, 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166494 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.765 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963734+0000", 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963734+0000", 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963735+0000", 2024-09-16T13:00:42.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6300000000000001e-07 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963736+0000", 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.748e-06 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963742+0000", 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.772e-06 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963746+0000", 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.608e-06 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042376+0000", 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078629839000000007 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042467+0000", 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1197000000000001e-05 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042565+0000", 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8616999999999994e-05 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166496 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 7102464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.963750+0000", 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.724793876, 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079033872000000005, 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166496 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963750+0000", 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963750+0000", 2024-09-16T13:00:42.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963750+0000", 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5599999999999995e-07 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963752+0000", 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8059999999999999e-06 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963758+0000", 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7189999999999998e-06 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963761+0000", 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6509999999999999e-06 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042598+0000", 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078836421000000004 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042686+0000", 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8010999999999995e-05 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042784+0000", 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7707999999999994e-05 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.773 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166498 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 7147520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.963781+0000", 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.724762261, 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079226252999999996, 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166498 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963781+0000", 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963781+0000", 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963782+0000", 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963784+0000", 2024-09-16T13:00:42.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.187e-06 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963791+0000", 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9820000000000002e-06 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963795+0000", 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1420000000000003e-06 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042814+0000", 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079018599999999994 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.042910+0000", 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6229000000000003e-05 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.043008+0000", 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7695000000000006e-05 2024-09-16T13:00:42.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:166499 4.f 4:f6ace142:::rbd_data.11b5c2ee3542.0000000000000006:1d [read 7192576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.963799+0000", 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.724744429999999, 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079436572999999996, 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:42.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 166499 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963799+0000", 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.963799+0000", 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.973831+0000", 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010031722999999999 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.973838+0000", 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3880000000000002e-06 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:42.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.979866+0000", 2024-09-16T13:00:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060278850000000002 2024-09-16T13:00:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.979881+0000", 2024-09-16T13:00:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4576000000000001e-05 2024-09-16T13:00:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:42.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.043038+0000", 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.063157074999999993 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.043128+0000", 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9901000000000005e-05 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:29.043236+0000", 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108025 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.782 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.783 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:42.784 INFO:teuthology.orchestra.run.smithi042.stdout:5309 write 0x972526e thru 0x9733b9d (0xe930 bytes) 2024-09-16T13:00:42.785 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:42.785 INFO:teuthology.orchestra.run.smithi042.stdout:5310 write 0x850af12 thru 0x8511dde (0x6ecd bytes) 2024-09-16T13:00:42.785 INFO:teuthology.orchestra.run.smithi042.stdout:5312 trunc from 0x9733b9e to 0x5b15742 2024-09-16T13:00:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:00:42.786 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 97, 2024-09-16T13:00:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7", 2024-09-16T13:00:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-16T12:58:33.966359+0000", 2024-09-16T13:00:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-16T13:00:41.389160+0000", 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-16T13:00:19.249119+0000", 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-16T12:59:39.968761+0000", 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5804032, 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-16T13:00:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 4, 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "luminous", 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "quincy", 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-16T13:00:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:58:44.284177+0000", 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-16T13:00:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "17", 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T13:00:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-16T13:00:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:58:51.733915+0000", 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T13:00:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T13:00:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "26", 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "26", 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 3, 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 24, 2024-09-16T13:00:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [ 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: 3 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": 3, 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": 3, 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T13:00:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T13:00:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "cache", 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:59:00.193651+0000", 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8201, 2024-09-16T13:00:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,incomplete_clones,selfmanaged_snaps", 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 4, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 4, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 4, 2024-09-16T13:00:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 4, 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 4, 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "30", 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "26", 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T13:00:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 3, 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 24, 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": 2, 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "writeback", 2024-09-16T13:00:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 250, 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bloom", 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "false_positive_probability": 0.050000000000000003, 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "target_size": 0, 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "seed": 0 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 60, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 8, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [ 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:00:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 4, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:59:09.779825+0000", 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T13:00:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T13:00:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "96", 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 36, 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 96, 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-16T13:00:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T13:00:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T13:00:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "0d8a8277-07cd-40f1-8c98-98b8b51d1689", 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:00:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-16T13:00:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6819/22186", 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6823/22186", 2024-09-16T13:00:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6830/22186", 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6826/22186", 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "6ce5996a-5e0e-4c9d-b268-27b34a14d830", 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:00:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/22184", 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/22184", 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/22184", 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/22184", 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "156042ab-909e-4368-b510-84018cb4354f", 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:00:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/22185", 2024-09-16T13:00:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/22185", 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/22185", 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/22185", 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b6c53823-446f-4413-a2ff-d2ebe8fe9c71", 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:00:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:00:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:00:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6818/22187", 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6822/22187", 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/22187", 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6827/22187", 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:00:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "278bfad1-010b-494e-ba61-bdf3935ca078", 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 0, 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 57, 2024-09-16T13:00:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 84, 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6816", 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6817", 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6818", 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6819", 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6822", 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6823", 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6820", 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6821", 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22339 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6817/22339", 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6819/22339", 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6823/22339", 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6821/22339", 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "exists" 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "ed1af73b-b108-4189-8223-f017e634e3e8", 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:00:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6808", 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6809", 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6810", 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6811", 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6814", 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6815", 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6812", 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6813", 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6809/22340", 2024-09-16T13:00:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6811/22340", 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6815/22340", 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6813/22340", 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "ce904ba8-02e9-4579-9635-b4b6c6de1a08", 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:00:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6824", 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6825", 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6826", 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T13:00:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6827", 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6830", 2024-09-16T13:00:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6831", 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6828", 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6829", 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22364 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6825/22364", 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6827/22364", 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6831/22364", 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6829/22364", 2024-09-16T13:00:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d911da1b-2725-46a8-bbfa-8968de47271f", 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:00:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 57, 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6800", 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6801", 2024-09-16T13:00:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6802", 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6803", 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6806", 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6807", 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6804", 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6805", 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6801/22341", 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6803/22341", 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6807/22341", 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6805/22341", 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.264463+0000", 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.214474+0000", 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:00:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.217927+0000", 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.435822+0000", 2024-09-16T13:00:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.451286+0000", 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 84 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.287403+0000", 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:00:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.440296+0000", 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:00:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.140630+0000", 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [ 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "4.4", 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "mappings": [ 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "from": 1, 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "to": 6 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-16T13:00:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:42.865 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:42.866 INFO:teuthology.orchestra.run.smithi042.stdout:5313 write 0x5998d9c thru 0x59a7801 (0xea66 bytes) 2024-09-16T13:00:42.867 INFO:tasks.thrashosds.thrasher:j is {'epoch': 97, 'fsid': 'c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7', 'created': '2024-09-16T12:58:33.966359+0000', 'modified': '2024-09-16T13:00:41.389160+0000', 'last_up_change': '2024-09-16T13:00:19.249119+0000', 'last_in_change': '2024-09-16T12:59:39.968761+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T12:58:44.284177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T12:58:51.733915+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T12:59:00.193651+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T12:59:09.779825+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 36, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': '0d8a8277-07cd-40f1-8c98-98b8b51d1689', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6817', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 22186}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6821', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 22186}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6830', 'nonce': 22186}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6825', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6826', 'nonce': 22186}]}, 'public_addr': '172.21.15.42:6819/22186', 'cluster_addr': '172.21.15.42:6823/22186', 'heartbeat_back_addr': '172.21.15.42:6830/22186', 'heartbeat_front_addr': '172.21.15.42:6826/22186', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6ce5996a-5e0e-4c9d-b268-27b34a14d830', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 22184}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 22184}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 22184}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 22184}]}, 'public_addr': '172.21.15.42:6809/22184', 'cluster_addr': '172.21.15.42:6811/22184', 'heartbeat_back_addr': '172.21.15.42:6815/22184', 'heartbeat_front_addr': '172.21.15.42:6813/22184', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '156042ab-909e-4368-b510-84018cb4354f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 22185}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 22185}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 22185}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 22185}]}, 'public_addr': '172.21.15.42:6801/22185', 'cluster_addr': '172.21.15.42:6803/22185', 'heartbeat_back_addr': '172.21.15.42:6807/22185', 'heartbeat_front_addr': '172.21.15.42:6805/22185', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b6c53823-446f-4413-a2ff-d2ebe8fe9c71', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6818', 'nonce': 22187}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6822', 'nonce': 22187}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6829', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 22187}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 22187}]}, 'public_addr': '172.21.15.42:6818/22187', 'cluster_addr': '172.21.15.42:6822/22187', 'heartbeat_back_addr': '172.21.15.42:6831/22187', 'heartbeat_front_addr': '172.21.15.42:6827/22187', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '278bfad1-010b-494e-ba61-bdf3935ca078', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 57, 'down_at': 84, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6816', 'nonce': 22339}, {'type': 'v1', 'addr': '172.21.15.136:6817', 'nonce': 22339}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6818', 'nonce': 22339}, {'type': 'v1', 'addr': '172.21.15.136:6819', 'nonce': 22339}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6822', 'nonce': 22339}, {'type': 'v1', 'addr': '172.21.15.136:6823', 'nonce': 22339}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6820', 'nonce': 22339}, {'type': 'v1', 'addr': '172.21.15.136:6821', 'nonce': 22339}]}, 'public_addr': '172.21.15.136:6817/22339', 'cluster_addr': '172.21.15.136:6819/22339', 'heartbeat_back_addr': '172.21.15.136:6823/22339', 'heartbeat_front_addr': '172.21.15.136:6821/22339', 'state': ['exists']}, {'osd': 5, 'uuid': 'ed1af73b-b108-4189-8223-f017e634e3e8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6809', 'nonce': 22340}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6810', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6811', 'nonce': 22340}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6814', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6815', 'nonce': 22340}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6813', 'nonce': 22340}]}, 'public_addr': '172.21.15.136:6809/22340', 'cluster_addr': '172.21.15.136:6811/22340', 'heartbeat_back_addr': '172.21.15.136:6815/22340', 'heartbeat_front_addr': '172.21.15.136:6813/22340', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ce904ba8-02e9-4579-9635-b4b6c6de1a08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6824', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.136:6825', 'nonce': 22364}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6826', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.136:6827', 'nonce': 22364}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6830', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.136:6831', 'nonce': 22364}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6828', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.136:6829', 'nonce': 22364}]}, 'public_addr': '172.21.15.136:6825/22364', 'cluster_addr': '172.21.15.136:6827/22364', 'heartbeat_back_addr': '172.21.15.136:6831/22364', 'heartbeat_front_addr': '172.21.15.136:6829/22364', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd911da1b-2725-46a8-bbfa-8968de47271f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6801', 'nonce': 22341}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6803', 'nonce': 22341}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6807', 'nonce': 22341}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6805', 'nonce': 22341}]}, 'public_addr': '172.21.15.136:6801/22341', 'cluster_addr': '172.21.15.136:6803/22341', 'heartbeat_back_addr': '172.21.15.136:6807/22341', 'heartbeat_front_addr': '172.21.15.136:6805/22341', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.264463+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.214474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.217927+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.435822+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.451286+0000', 'dead_epoch': 84}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.287403+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.440296+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.140630+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 1, 'to': 6}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T13:00:42.867 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:00:42.868 INFO:teuthology.orchestra.run.smithi042.stdout:5314 write 0x756ed24 thru 0x7575de1 (0x70be bytes) 2024-09-16T13:00:42.868 INFO:teuthology.orchestra.run.smithi042.stdout:5315 write 0x5755193 thru 0x5759031 (0x3e9f bytes) 2024-09-16T13:00:42.868 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:42.776+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:42.869 INFO:teuthology.orchestra.run.smithi042.stdout:5317 punch from 0x5fb89d9 to 0x5fc3e05, (0xb42c bytes) 2024-09-16T13:00:42.869 INFO:teuthology.orchestra.run.smithi042.stdout:5318 trunc from 0x7575de2 to 0x82cb242 2024-09-16T13:00:42.869 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:42.869 INFO:teuthology.orchestra.run.smithi042.stdout:5320 punch from 0x5701521 to 0x5701b21, (0x600 bytes) 2024-09-16T13:00:42.870 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:42.870 INFO:teuthology.orchestra.run.smithi042.stdout:5323 read 0x1b8062f thru 0x1b88fe9 (0x89bb bytes) 2024-09-16T13:00:42.870 INFO:teuthology.orchestra.run.smithi042.stdout:5324 write 0x21d57ef thru 0x21dbfdc (0x67ee bytes) 2024-09-16T13:00:42.870 INFO:teuthology.orchestra.run.smithi042.stdout:5325 read 0x1573c81 thru 0x1578e45 (0x51c5 bytes) 2024-09-16T13:00:42.870 INFO:teuthology.orchestra.run.smithi042.stdout:5326 clone 34 order 23 su 1048576 sc 8 2024-09-16T13:00:42.885 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:42.877+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:42.986 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:42.980+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:43.012 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:43.013 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:43.013 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:43.013 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:43.022 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:43.087 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:43.081+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:43.123 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:43.123 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:43.123 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:43.123 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:43.123 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:43.132 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:43.188 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:43.181+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:43.215 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:00:43.219 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:00:43.231 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-16T13:00:42.264007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1159983370,"num_objects":2181,"num_object_clones":382,"num_object_copies":4362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2178,"num_whiteouts":1146,"num_read":281324,"num_read_kb":565559,"num_write":15914,"num_write_kb":269410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":6035857,"num_keys_recovered":111,"num_objects_omap":36,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13633,"ondisk_log_size":13633,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2516884,"kb_used_data":42344,"kb_used_omap":13,"kb_used_meta":2474418,"kb_avail":752457836,"statfs":{"total":773094113280,"available":770516824064,"internally_reserved":0,"allocated":43360256,"data_stored":286250417,"data_compressed":6837823,"data_compressed_allocated":19038208,"data_compressed_original":269668024,"omap_allocated":13632,"internal_metadata":2533804736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":59953433,"num_objects":640,"num_object_clones":27,"num_object_copies":1280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":639,"num_whiteouts":496,"num_read":18903,"num_read_kb":44544,"num_write":1976,"num_write_kb":19454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.900755"},"pg_stats":[{"pgid":"4.1f","version":"96'221","reported_seq":2812,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.110184+0000","last_change":"2024-09-16T12:59:13.789786+0000","last_active":"2024-09-16T13:00:40.110184+0000","last_peered":"2024-09-16T13:00:40.110184+0000","last_clean":"2024-09-16T13:00:40.110184+0000","last_became_active":"2024-09-16T12:59:11.766798+0000","last_became_peered":"2024-09-16T12:59:11.766798+0000","last_unstale":"2024-09-16T13:00:40.110184+0000","last_undegraded":"2024-09-16T13:00:40.110184+0000","last_fullsized":"2024-09-16T13:00:40.110184+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:22.588888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":20966621,"num_objects":62,"num_object_clones":14,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":30,"num_read":2816,"num_read_kb":16274,"num_write":249,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"96'336","reported_seq":15709,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.080430+0000","last_change":"2024-09-16T12:59:14.030480+0000","last_active":"2024-09-16T13:00:40.080430+0000","last_peered":"2024-09-16T13:00:40.080430+0000","last_clean":"2024-09-16T13:00:40.080430+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:00:40.080430+0000","last_undegraded":"2024-09-16T13:00:40.080430+0000","last_fullsized":"2024-09-16T13:00:40.080430+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049602200000000004,"stat_sum":{"num_bytes":40502333,"num_objects":86,"num_object_clones":17,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":45,"num_read":15930,"num_read_kb":15953,"num_write":345,"num_write_kb":7426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"96'283","reported_seq":2627,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.082326+0000","last_change":"2024-09-16T12:59:41.178204+0000","last_active":"2024-09-16T13:00:40.082326+0000","last_peered":"2024-09-16T13:00:40.082326+0000","last_clean":"2024-09-16T13:00:40.082326+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:00:40.082326+0000","last_undegraded":"2024-09-16T13:00:40.082326+0000","last_fullsized":"2024-09-16T13:00:40.082326+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012332339999999999,"stat_sum":{"num_bytes":26736354,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":2328,"num_read_kb":12855,"num_write":231,"num_write_kb":10613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"96'996","reported_seq":4796,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.235599+0000","last_change":"2024-09-16T12:59:41.030038+0000","last_active":"2024-09-16T13:00:40.235599+0000","last_peered":"2024-09-16T13:00:40.235599+0000","last_clean":"2024-09-16T13:00:40.235599+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:00:40.235599+0000","last_undegraded":"2024-09-16T13:00:40.235599+0000","last_fullsized":"2024-09-16T13:00:40.235599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":996,"log_dups_size":0,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015138090000000001,"stat_sum":{"num_bytes":22232251,"num_objects":78,"num_object_clones":10,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":45,"num_read":7220,"num_read_kb":10609,"num_write":1359,"num_write_kb":5416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"96'367","reported_seq":20992,"reported_epoch":96,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:40.223969+0000","last_change":"2024-09-16T13:00:20.273717+0000","last_active":"2024-09-16T13:00:40.223969+0000","last_peered":"2024-09-16T13:00:40.223969+0000","last_clean":"2024-09-16T13:00:18.903627+0000","last_became_active":"2024-09-16T13:00:20.273717+0000","last_became_peered":"2024-09-16T13:00:20.273717+0000","last_unstale":"2024-09-16T13:00:40.223969+0000","last_undegraded":"2024-09-16T13:00:20.272222+0000","last_fullsized":"2024-09-16T13:00:20.272074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:50.116815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002573915,"stat_sum":{"num_bytes":51579248,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":42,"num_read":22047,"num_read_kb":44031,"num_write":350,"num_write_kb":7615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":74}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"96'535","reported_seq":4658,"reported_epoch":96,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:40.253153+0000","last_change":"2024-09-16T13:00:20.273695+0000","last_active":"2024-09-16T13:00:40.253153+0000","last_peered":"2024-09-16T13:00:40.253153+0000","last_clean":"2024-09-16T13:00:17.071817+0000","last_became_active":"2024-09-16T13:00:20.273695+0000","last_became_peered":"2024-09-16T13:00:20.273695+0000","last_unstale":"2024-09-16T13:00:40.253153+0000","last_undegraded":"2024-09-16T13:00:20.272075+0000","last_fullsized":"2024-09-16T13:00:20.271923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:03.132002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002565964,"stat_sum":{"num_bytes":46793237,"num_objects":80,"num_object_clones":14,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":39,"num_read":5767,"num_read_kb":14009,"num_write":776,"num_write_kb":8109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":80}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"96'294","reported_seq":4372,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.130675+0000","last_change":"2024-09-16T12:59:13.770391+0000","last_active":"2024-09-16T13:00:40.130675+0000","last_peered":"2024-09-16T13:00:40.130675+0000","last_clean":"2024-09-16T13:00:40.130675+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:00:40.130675+0000","last_undegraded":"2024-09-16T13:00:40.130675+0000","last_fullsized":"2024-09-16T13:00:40.130675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013782849999999999,"stat_sum":{"num_bytes":20466743,"num_objects":70,"num_object_clones":16,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":37,"num_read":4107,"num_read_kb":17383,"num_write":275,"num_write_kb":13357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"96'322","reported_seq":3458,"reported_epoch":96,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:40.140574+0000","last_change":"2024-09-16T13:00:20.278527+0000","last_active":"2024-09-16T13:00:40.140574+0000","last_peered":"2024-09-16T13:00:40.140574+0000","last_clean":"2024-09-16T13:00:18.267176+0000","last_became_active":"2024-09-16T13:00:20.278527+0000","last_became_peered":"2024-09-16T13:00:20.278527+0000","last_unstale":"2024-09-16T13:00:40.140574+0000","last_undegraded":"2024-09-16T13:00:20.277954+0000","last_fullsized":"2024-09-16T13:00:20.277851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:51:12.520578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093274999999999999,"stat_sum":{"num_bytes":40794727,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":34,"num_read":3680,"num_read_kb":15998,"num_write":385,"num_write_kb":9263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":74}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"96'751","reported_seq":5134,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.167221+0000","last_change":"2024-09-16T12:59:13.770695+0000","last_active":"2024-09-16T13:00:40.167221+0000","last_peered":"2024-09-16T13:00:40.167221+0000","last_clean":"2024-09-16T13:00:40.167221+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:00:40.167221+0000","last_undegraded":"2024-09-16T13:00:40.167221+0000","last_fullsized":"2024-09-16T13:00:40.167221+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001481132,"stat_sum":{"num_bytes":64333597,"num_objects":84,"num_object_clones":12,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":42,"num_read":8585,"num_read_kb":27459,"num_write":937,"num_write_kb":13619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"96'311","reported_seq":2000,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.230137+0000","last_change":"2024-09-16T12:59:14.030762+0000","last_active":"2024-09-16T13:00:40.230137+0000","last_peered":"2024-09-16T13:00:40.230137+0000","last_clean":"2024-09-16T13:00:40.230137+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:00:40.230137+0000","last_undegraded":"2024-09-16T13:00:40.230137+0000","last_fullsized":"2024-09-16T13:00:40.230137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685533,"stat_sum":{"num_bytes":25927058,"num_objects":71,"num_object_clones":10,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":44,"num_read":2784,"num_read_kb":9717,"num_write":269,"num_write_kb":2161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"96'321","reported_seq":15479,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.155057+0000","last_change":"2024-09-16T12:59:13.796422+0000","last_active":"2024-09-16T13:00:40.155057+0000","last_peered":"2024-09-16T13:00:40.155057+0000","last_clean":"2024-09-16T13:00:40.155057+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:00:40.155057+0000","last_undegraded":"2024-09-16T13:00:40.155057+0000","last_fullsized":"2024-09-16T13:00:40.155057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055065799999999999,"stat_sum":{"num_bytes":37968482,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":40,"num_read":15687,"num_read_kb":14686,"num_write":357,"num_write_kb":5131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"96'453","reported_seq":20624,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.111665+0000","last_change":"2024-09-16T12:59:41.127844+0000","last_active":"2024-09-16T13:00:40.111665+0000","last_peered":"2024-09-16T13:00:40.111665+0000","last_clean":"2024-09-16T13:00:40.111665+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:00:40.111665+0000","last_undegraded":"2024-09-16T13:00:40.111665+0000","last_fullsized":"2024-09-16T13:00:40.111665+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013482349999999999,"stat_sum":{"num_bytes":67286407,"num_objects":78,"num_object_clones":18,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":39,"num_read":21108,"num_read_kb":31803,"num_write":504,"num_write_kb":15556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"96'364","reported_seq":5414,"reported_epoch":96,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:40.154559+0000","last_change":"2024-09-16T13:00:20.278920+0000","last_active":"2024-09-16T13:00:40.154559+0000","last_peered":"2024-09-16T13:00:40.154559+0000","last_clean":"2024-09-16T13:00:16.244289+0000","last_became_active":"2024-09-16T13:00:20.278920+0000","last_became_peered":"2024-09-16T13:00:20.278920+0000","last_unstale":"2024-09-16T13:00:40.154559+0000","last_undegraded":"2024-09-16T13:00:20.278208+0000","last_fullsized":"2024-09-16T13:00:20.277996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:09.910764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024518000000000001,"stat_sum":{"num_bytes":52792047,"num_objects":66,"num_object_clones":11,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":6027,"num_read_kb":26863,"num_write":365,"num_write_kb":40291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":66}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"94'259","reported_seq":2542,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T13:00:38.097625+0000","last_change":"2024-09-16T12:59:41.051975+0000","last_active":"2024-09-16T13:00:38.097625+0000","last_peered":"2024-09-16T13:00:38.097625+0000","last_clean":"2024-09-16T13:00:38.097625+0000","last_became_active":"2024-09-16T12:59:40.994472+0000","last_became_peered":"2024-09-16T12:59:40.994472+0000","last_unstale":"2024-09-16T13:00:38.097625+0000","last_undegraded":"2024-09-16T13:00:38.097625+0000","last_fullsized":"2024-09-16T13:00:38.097625+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:54.091219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061900799999999999,"stat_sum":{"num_bytes":39159932,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":29,"num_read":2762,"num_read_kb":14250,"num_write":275,"num_write_kb":3004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":162626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"96'261","reported_seq":15841,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.241018+0000","last_change":"2024-09-16T12:59:13.770311+0000","last_active":"2024-09-16T13:00:40.241018+0000","last_peered":"2024-09-16T13:00:40.241018+0000","last_clean":"2024-09-16T13:00:40.241018+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:00:40.241018+0000","last_undegraded":"2024-09-16T13:00:40.241018+0000","last_fullsized":"2024-09-16T13:00:40.241018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413508,"stat_sum":{"num_bytes":59261740,"num_objects":66,"num_object_clones":9,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":42,"num_read":15784,"num_read_kb":24914,"num_write":339,"num_write_kb":9461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"96'255","reported_seq":3403,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.173215+0000","last_change":"2024-09-16T12:59:41.065829+0000","last_active":"2024-09-16T13:00:40.173215+0000","last_peered":"2024-09-16T13:00:40.173215+0000","last_clean":"2024-09-16T13:00:40.173215+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:00:40.173215+0000","last_undegraded":"2024-09-16T13:00:40.173215+0000","last_fullsized":"2024-09-16T13:00:40.173215+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047952,"stat_sum":{"num_bytes":15282073,"num_objects":72,"num_object_clones":10,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":48,"num_read":3292,"num_read_kb":13497,"num_write":256,"num_write_kb":2076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":86,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T13:00:38.379905+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T13:00:38.379905+0000","last_peered":"2024-09-16T13:00:38.379905+0000","last_clean":"2024-09-16T13:00:38.379905+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T13:00:38.379905+0000","last_undegraded":"2024-09-16T13:00:38.379905+0000","last_fullsized":"2024-09-16T13:00:38.379905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"96'329","reported_seq":2531,"reported_epoch":96,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:40.257061+0000","last_change":"2024-09-16T13:00:20.275170+0000","last_active":"2024-09-16T13:00:40.257061+0000","last_peered":"2024-09-16T13:00:40.257061+0000","last_clean":"2024-09-16T13:00:16.243839+0000","last_became_active":"2024-09-16T13:00:20.275170+0000","last_became_peered":"2024-09-16T13:00:20.275170+0000","last_unstale":"2024-09-16T13:00:40.257061+0000","last_undegraded":"2024-09-16T13:00:20.274206+0000","last_fullsized":"2024-09-16T13:00:20.273957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:28.471020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069048899999999997,"stat_sum":{"num_bytes":29299633,"num_objects":56,"num_object_clones":5,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":33,"num_read":3293,"num_read_kb":2532,"num_write":276,"num_write_kb":3485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":56}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T13:00:36.373605+0000","last_change":"2024-09-16T12:59:40.993864+0000","last_active":"2024-09-16T13:00:36.373605+0000","last_peered":"2024-09-16T13:00:36.373605+0000","last_clean":"2024-09-16T13:00:36.373605+0000","last_became_active":"2024-09-16T12:59:40.992958+0000","last_became_peered":"2024-09-16T12:59:40.992958+0000","last_unstale":"2024-09-16T13:00:36.373605+0000","last_undegraded":"2024-09-16T13:00:36.373605+0000","last_fullsized":"2024-09-16T13:00:36.373605+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:44.409493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"96'526","reported_seq":5368,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.142961+0000","last_change":"2024-09-16T12:59:13.769781+0000","last_active":"2024-09-16T13:00:40.142961+0000","last_peered":"2024-09-16T13:00:40.142961+0000","last_clean":"2024-09-16T13:00:40.142961+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:00:40.142961+0000","last_undegraded":"2024-09-16T13:00:40.142961+0000","last_fullsized":"2024-09-16T13:00:40.142961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010481990000000001,"stat_sum":{"num_bytes":30373221,"num_objects":90,"num_object_clones":16,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":54,"num_read":7293,"num_read_kb":14100,"num_write":471,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":87,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T13:00:36.382694+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:00:36.382694+0000","last_peered":"2024-09-16T13:00:36.382694+0000","last_clean":"2024-09-16T13:00:36.382694+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:00:36.382694+0000","last_undegraded":"2024-09-16T13:00:36.382694+0000","last_fullsized":"2024-09-16T13:00:36.382694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"96'375","reported_seq":1285,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.093537+0000","last_change":"2024-09-16T12:59:14.030767+0000","last_active":"2024-09-16T13:00:40.093537+0000","last_peered":"2024-09-16T13:00:40.093537+0000","last_clean":"2024-09-16T13:00:40.093537+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:00:40.093537+0000","last_undegraded":"2024-09-16T13:00:40.093537+0000","last_fullsized":"2024-09-16T13:00:40.093537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000670212,"stat_sum":{"num_bytes":5740046,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":27,"num_read":1320,"num_read_kb":3529,"num_write":281,"num_write_kb":5794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T13:00:36.373397+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:00:36.373397+0000","last_peered":"2024-09-16T13:00:36.373397+0000","last_clean":"2024-09-16T13:00:36.373397+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:00:36.373397+0000","last_undegraded":"2024-09-16T13:00:36.373397+0000","last_fullsized":"2024-09-16T13:00:36.373397+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"94'390","reported_seq":15841,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T13:00:38.096353+0000","last_change":"2024-09-16T12:59:41.165565+0000","last_active":"2024-09-16T13:00:38.096353+0000","last_peered":"2024-09-16T13:00:38.096353+0000","last_clean":"2024-09-16T13:00:38.096353+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:00:38.096353+0000","last_undegraded":"2024-09-16T13:00:38.096353+0000","last_fullsized":"2024-09-16T13:00:38.096353+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045226000000000001,"stat_sum":{"num_bytes":21675815,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":17056,"num_read_kb":18014,"num_write":517,"num_write_kb":5895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":95,"state":"active+undersized","last_fresh":"2024-09-16T13:00:38.430630+0000","last_change":"2024-09-16T13:00:20.276712+0000","last_active":"2024-09-16T13:00:38.430630+0000","last_peered":"2024-09-16T13:00:38.430630+0000","last_clean":"2024-09-16T12:59:00.662491+0000","last_became_active":"2024-09-16T13:00:20.276712+0000","last_became_peered":"2024-09-16T13:00:20.276712+0000","last_unstale":"2024-09-16T13:00:38.430630+0000","last_undegraded":"2024-09-16T13:00:38.430630+0000","last_fullsized":"2024-09-16T13:00:20.275962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:21.793903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":115,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T13:00:38.374439+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:00:38.374439+0000","last_peered":"2024-09-16T13:00:38.374439+0000","last_clean":"2024-09-16T13:00:38.374439+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:00:38.374439+0000","last_undegraded":"2024-09-16T13:00:38.374439+0000","last_fullsized":"2024-09-16T13:00:38.374439+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"96'329","reported_seq":17037,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.154478+0000","last_change":"2024-09-16T12:59:56.109592+0000","last_active":"2024-09-16T13:00:40.154478+0000","last_peered":"2024-09-16T13:00:40.154478+0000","last_clean":"2024-09-16T13:00:40.154478+0000","last_became_active":"2024-09-16T12:59:56.105790+0000","last_became_peered":"2024-09-16T12:59:56.105790+0000","last_unstale":"2024-09-16T13:00:40.154478+0000","last_undegraded":"2024-09-16T13:00:40.154478+0000","last_fullsized":"2024-09-16T13:00:40.154478+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:32.594440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077242,"stat_sum":{"num_bytes":20191241,"num_objects":71,"num_object_clones":15,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":35,"num_read":18087,"num_read_kb":20712,"num_write":382,"num_write_kb":7913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T13:00:36.373155+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:00:36.373155+0000","last_peered":"2024-09-16T13:00:36.373155+0000","last_clean":"2024-09-16T13:00:36.373155+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:00:36.373155+0000","last_undegraded":"2024-09-16T13:00:36.373155+0000","last_fullsized":"2024-09-16T13:00:36.373155+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":90,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T13:00:38.372431+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:00:38.372431+0000","last_peered":"2024-09-16T13:00:38.372431+0000","last_clean":"2024-09-16T13:00:38.372431+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:00:38.372431+0000","last_undegraded":"2024-09-16T13:00:38.372431+0000","last_fullsized":"2024-09-16T13:00:38.372431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"94'280","reported_seq":1504,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T13:00:38.165698+0000","last_change":"2024-09-16T12:59:41.112327+0000","last_active":"2024-09-16T13:00:38.165698+0000","last_peered":"2024-09-16T13:00:38.165698+0000","last_clean":"2024-09-16T13:00:38.165698+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:00:38.165698+0000","last_undegraded":"2024-09-16T13:00:38.165698+0000","last_fullsized":"2024-09-16T13:00:38.165698+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064957100000000005,"stat_sum":{"num_bytes":55052986,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":20,"num_read":1848,"num_read_kb":18041,"num_write":327,"num_write_kb":7794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":84,"reported_epoch":95,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:38.430382+0000","last_change":"2024-09-16T13:00:20.276924+0000","last_active":"2024-09-16T13:00:38.430382+0000","last_peered":"2024-09-16T13:00:38.430382+0000","last_clean":"2024-09-16T13:00:18.578645+0000","last_became_active":"2024-09-16T13:00:20.276924+0000","last_became_peered":"2024-09-16T13:00:20.276924+0000","last_unstale":"2024-09-16T13:00:38.430382+0000","last_undegraded":"2024-09-16T13:00:20.276013+0000","last_fullsized":"2024-09-16T13:00:20.275883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:45.681853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T13:00:36.382344+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:00:36.382344+0000","last_peered":"2024-09-16T13:00:36.382344+0000","last_clean":"2024-09-16T13:00:36.382344+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:00:36.382344+0000","last_undegraded":"2024-09-16T13:00:36.382344+0000","last_fullsized":"2024-09-16T13:00:36.382344+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"96'305","reported_seq":4983,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.107453+0000","last_change":"2024-09-16T12:59:41.090757+0000","last_active":"2024-09-16T13:00:40.107453+0000","last_peered":"2024-09-16T13:00:40.107453+0000","last_clean":"2024-09-16T13:00:40.107453+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:00:40.107453+0000","last_undegraded":"2024-09-16T13:00:40.107453+0000","last_fullsized":"2024-09-16T13:00:40.107453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094804399999999999,"stat_sum":{"num_bytes":33648800,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":34,"num_read":4959,"num_read_kb":11259,"num_write":252,"num_write_kb":6490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"94'2","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T13:00:38.374415+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:00:38.374415+0000","last_peered":"2024-09-16T13:00:38.374415+0000","last_clean":"2024-09-16T13:00:38.374415+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:00:38.374415+0000","last_undegraded":"2024-09-16T13:00:38.374415+0000","last_fullsized":"2024-09-16T13:00:38.374415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:53:17.807916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":88,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T13:00:38.374459+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:00:38.374459+0000","last_peered":"2024-09-16T13:00:38.374459+0000","last_clean":"2024-09-16T13:00:38.374459+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:00:38.374459+0000","last_undegraded":"2024-09-16T13:00:38.374459+0000","last_fullsized":"2024-09-16T13:00:38.374459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":175,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T13:00:36.382568+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:00:36.382568+0000","last_peered":"2024-09-16T13:00:36.382568+0000","last_clean":"2024-09-16T13:00:36.382568+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:00:36.382568+0000","last_undegraded":"2024-09-16T13:00:36.382568+0000","last_fullsized":"2024-09-16T13:00:36.382568+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"96'292","reported_seq":2442,"reported_epoch":96,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:40.143890+0000","last_change":"2024-09-16T13:00:20.279956+0000","last_active":"2024-09-16T13:00:40.143890+0000","last_peered":"2024-09-16T13:00:40.143890+0000","last_clean":"2024-09-16T13:00:18.737959+0000","last_became_active":"2024-09-16T13:00:20.279956+0000","last_became_peered":"2024-09-16T13:00:20.279956+0000","last_unstale":"2024-09-16T13:00:40.143890+0000","last_undegraded":"2024-09-16T13:00:20.278283+0000","last_fullsized":"2024-09-16T13:00:20.278102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:38.614832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046786199999999999,"stat_sum":{"num_bytes":13859205,"num_objects":87,"num_object_clones":17,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":87,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":47,"num_read":2649,"num_read_kb":7708,"num_write":270,"num_write_kb":5181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":87}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"96'496","reported_seq":17240,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.133638+0000","last_change":"2024-09-16T12:59:13.796421+0000","last_active":"2024-09-16T13:00:40.133638+0000","last_peered":"2024-09-16T13:00:40.133638+0000","last_clean":"2024-09-16T13:00:40.133638+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:00:40.133638+0000","last_undegraded":"2024-09-16T13:00:40.133638+0000","last_fullsized":"2024-09-16T13:00:40.133638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067313300000000002,"stat_sum":{"num_bytes":61324240,"num_objects":78,"num_object_clones":13,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":34,"num_read":18247,"num_read_kb":24161,"num_write":548,"num_write_kb":8691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"94'226","reported_seq":2229,"reported_epoch":94,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:38.008411+0000","last_change":"2024-09-16T13:00:20.278269+0000","last_active":"2024-09-16T13:00:38.008411+0000","last_peered":"2024-09-16T13:00:38.008411+0000","last_clean":"2024-09-16T13:00:16.243819+0000","last_became_active":"2024-09-16T13:00:20.278269+0000","last_became_peered":"2024-09-16T13:00:20.278269+0000","last_unstale":"2024-09-16T13:00:38.008411+0000","last_undegraded":"2024-09-16T13:00:20.276052+0000","last_fullsized":"2024-09-16T13:00:20.275885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:43.307424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011000280000000001,"stat_sum":{"num_bytes":13745800,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":2097,"num_read_kb":4414,"num_write":240,"num_write_kb":7058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":16420,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":36}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"96'948","reported_seq":5916,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.275904+0000","last_change":"2024-09-16T12:59:44.030680+0000","last_active":"2024-09-16T13:00:40.275904+0000","last_peered":"2024-09-16T13:00:40.275904+0000","last_clean":"2024-09-16T13:00:40.275904+0000","last_became_active":"2024-09-16T12:59:44.028122+0000","last_became_peered":"2024-09-16T12:59:44.028122+0000","last_unstale":"2024-09-16T13:00:40.275904+0000","last_undegraded":"2024-09-16T13:00:40.275904+0000","last_fullsized":"2024-09-16T13:00:40.275904+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":948,"log_dups_size":0,"ondisk_log_size":948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:27:27.517524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078078700000000002,"stat_sum":{"num_bytes":47291745,"num_objects":66,"num_object_clones":12,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":34,"num_read":9446,"num_read_kb":18260,"num_write":1486,"num_write_kb":4567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33178,"num_keys_recovered":22,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"96'807","reported_seq":6099,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T13:00:40.085987+0000","last_change":"2024-09-16T12:59:41.151398+0000","last_active":"2024-09-16T13:00:40.085987+0000","last_peered":"2024-09-16T13:00:40.085987+0000","last_clean":"2024-09-16T13:00:40.085987+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:00:40.085987+0000","last_undegraded":"2024-09-16T13:00:40.085987+0000","last_fullsized":"2024-09-16T13:00:40.085987+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053245600000000005,"stat_sum":{"num_bytes":23071209,"num_objects":78,"num_object_clones":11,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":40,"num_read":8516,"num_read_kb":18650,"num_write":1274,"num_write_kb":6767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"96'416","reported_seq":4284,"reported_epoch":96,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:40.116205+0000","last_change":"2024-09-16T13:00:20.274232+0000","last_active":"2024-09-16T13:00:40.116205+0000","last_peered":"2024-09-16T13:00:40.116205+0000","last_clean":"2024-09-16T13:00:16.671540+0000","last_became_active":"2024-09-16T13:00:20.274232+0000","last_became_peered":"2024-09-16T13:00:20.274232+0000","last_unstale":"2024-09-16T13:00:40.116205+0000","last_undegraded":"2024-09-16T13:00:20.272352+0000","last_fullsized":"2024-09-16T13:00:20.272219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:18.892028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020639640000000002,"stat_sum":{"num_bytes":38200999,"num_objects":66,"num_object_clones":12,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":39,"num_read":4974,"num_read_kb":14208,"num_write":412,"num_write_kb":3691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":66}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"96'280","reported_seq":2731,"reported_epoch":96,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:40.115919+0000","last_change":"2024-09-16T13:00:20.271516+0000","last_active":"2024-09-16T13:00:40.115919+0000","last_peered":"2024-09-16T13:00:40.115919+0000","last_clean":"2024-09-16T13:00:18.254553+0000","last_became_active":"2024-09-16T13:00:20.271516+0000","last_became_peered":"2024-09-16T13:00:20.271516+0000","last_unstale":"2024-09-16T13:00:40.115919+0000","last_undegraded":"2024-09-16T13:00:20.270680+0000","last_fullsized":"2024-09-16T13:00:20.270585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:48.815343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001758094,"stat_sum":{"num_bytes":41233577,"num_objects":61,"num_object_clones":8,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":61,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":37,"num_read":3112,"num_read_kb":12208,"num_write":304,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":61}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"96'715","reported_seq":3047,"reported_epoch":96,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:40.035804+0000","last_change":"2024-09-16T13:00:20.279463+0000","last_active":"2024-09-16T13:00:40.035804+0000","last_peered":"2024-09-16T13:00:40.035804+0000","last_clean":"2024-09-16T13:00:18.910076+0000","last_became_active":"2024-09-16T13:00:20.279463+0000","last_became_peered":"2024-09-16T13:00:20.279463+0000","last_unstale":"2024-09-16T13:00:40.035804+0000","last_undegraded":"2024-09-16T13:00:20.278840+0000","last_fullsized":"2024-09-16T13:00:20.278665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:46.024924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637831,"stat_sum":{"num_bytes":39185305,"num_objects":71,"num_object_clones":14,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":71,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":36,"num_read":4682,"num_read_kb":6295,"num_write":876,"num_write_kb":7931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":71}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"94'512","reported_seq":32551,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T13:00:38.261765+0000","last_change":"2024-09-16T12:59:41.023359+0000","last_active":"2024-09-16T13:00:38.261765+0000","last_peered":"2024-09-16T13:00:38.261765+0000","last_clean":"2024-09-16T13:00:38.261765+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:00:38.261765+0000","last_undegraded":"2024-09-16T13:00:38.261765+0000","last_fullsized":"2024-09-16T13:00:38.261765+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053943399999999999,"stat_sum":{"num_bytes":53416228,"num_objects":70,"num_object_clones":16,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":27,"num_read":33755,"num_read_kb":61112,"num_write":622,"num_write_kb":21232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1159392900,"num_objects":2174,"num_object_clones":382,"num_object_copies":4348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2174,"num_whiteouts":1144,"num_read":281258,"num_read_kb":565504,"num_write":15810,"num_write_kb":267779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":6035819,"num_keys_recovered":111,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34447360,"data_stored":280814184,"data_compressed":6818946,"data_compressed_allocated":18952192,"data_compressed_original":268496568,"omap_allocated":0,"internal_metadata":0},"log_size":13555,"ondisk_log_size":13555,"up":54,"acting":54,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":6,"acting":6,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320620,"kb_used_data":6628,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94051220,"statfs":{"total":96636764160,"available":96308449280,"internally_reserved":0,"allocated":6787072,"data_stored":49563016,"data_compressed":1226019,"data_compressed_allocated":3010560,"data_compressed_original":46871818,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.1459999999999999}]},{"osd":3,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.68700000000000006}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.1779999999999999}]},{"osd":5,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.71599999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509467,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313752,"kb_used_data":3920,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058088,"statfs":{"total":96636764160,"available":96315482112,"internally_reserved":0,"allocated":4014080,"data_stored":24663569,"data_compressed":470021,"data_compressed_allocated":1634304,"data_compressed_original":23088734,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.378},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.1259999999999999}]},{"osd":1,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.099}]},{"osd":2,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.544},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":1.5980000000000001}]},{"osd":4,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.1619999999999999}]},{"osd":6,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.036}]},{"osd":7,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.97499999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313200,"kb_used_data":3364,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058640,"statfs":{"total":96636764160,"available":96316047360,"internally_reserved":0,"allocated":3444736,"data_stored":17681952,"data_compressed":390745,"data_compressed_allocated":1073152,"data_compressed_original":16002788,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":1.204},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.626}]},{"osd":3,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.53800000000000003}]},{"osd":5,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.51700000000000002}]},{"osd":7,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.84399999999999997}]}]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314856,"kb_used_data":9184,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94056984,"statfs":{"total":96636764160,"available":96314351616,"internally_reserved":0,"allocated":9404416,"data_stored":72635742,"data_compressed":1620047,"data_compressed_allocated":4812800,"data_compressed_original":69331668,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.33900000000000002}]},{"osd":1,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.379},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":1.528}]},{"osd":2,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.6220000000000001}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.21},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.238}]},{"osd":5,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.57799999999999996},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.6619999999999999}]},{"osd":6,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.64300000000000002}]},{"osd":7,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.55500000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313568,"kb_used_data":3736,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058272,"statfs":{"total":96636764160,"available":96315670528,"internally_reserved":0,"allocated":3825664,"data_stored":22215836,"data_compressed":667130,"data_compressed_allocated":1470464,"data_compressed_original":20665386,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.36499999999999999}]},{"osd":1,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.47699999999999998}]},{"osd":4,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.161},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.78000000000000003}]},{"osd":5,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.65600000000000003}]},{"osd":6,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.67700000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313088,"kb_used_data":3256,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058752,"statfs":{"total":96636764160,"available":96316162048,"internally_reserved":0,"allocated":3334144,"data_stored":15440795,"data_compressed":535378,"data_compressed_allocated":1130496,"data_compressed_original":14130646,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.378},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.437}]},{"osd":2,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.35499999999999998}]},{"osd":3,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.53600000000000003}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.626}]},{"osd":6,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.747}]},{"osd":7,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.56100000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314756,"kb_used_data":4888,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94057084,"statfs":{"total":96636764160,"available":96314454016,"internally_reserved":0,"allocated":5005312,"data_stored":32108207,"data_compressed":748966,"data_compressed_allocated":2265088,"data_compressed_original":30310730,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":6.6139999999999999,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":1.3180000000000001}]},{"osd":1,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.6710000000000003,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":1.681},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.57299999999999995}]},{"osd":2,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":4.1449999999999996,"5min":4.1449999999999996,"15min":4.1449999999999996},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.0979999999999999,"5min":4.0979999999999999,"15min":4.0979999999999999},"last":1.2549999999999999}]},{"osd":3,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":6.3570000000000002,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.4550000000000001}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.7889999999999997,"5min":4.7889999999999997,"15min":4.7889999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.1539999999999999}]},{"osd":5,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":6.282,"5min":6.282,"15min":6.282},"last":1.75},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":6.3339999999999996,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.85899999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313044,"kb_used_data":7368,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94058796,"statfs":{"total":96636764160,"available":96316207104,"internally_reserved":0,"allocated":7544832,"data_stored":51941300,"data_compressed":1179517,"data_compressed_allocated":3641344,"data_compressed_original":49266254,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.46800000000000003}]},{"osd":2,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.024,"5min":3.024,"15min":3.024},"last":0.65200000000000002}]},{"osd":3,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":5.9390000000000001,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.873},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.073}]},{"osd":5,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":9.4920000000000009,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.59799999999999998}]},{"osd":6,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.52400000000000002}]},{"osd":7,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2146304,"data_stored":14302467,"data_compressed":526040,"data_compressed_allocated":1093632,"data_compressed_original":13557206,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6430720,"data_stored":51400124,"data_compressed":1179517,"data_compressed_allocated":3641344,"data_compressed_original":49266254,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2695168,"data_stored":21667876,"data_compressed":667130,"data_compressed_allocated":1470464,"data_compressed_original":20665386,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8269824,"data_stored":72087699,"data_compressed":1620047,"data_compressed_allocated":4812800,"data_compressed_original":69331668,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2543616,"data_stored":17249027,"data_compressed":390544,"data_compressed_allocated":1060864,"data_compressed_original":15978212,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2875392,"data_stored":24115507,"data_compressed":470021,"data_compressed_allocated":1634304,"data_compressed_original":23088734,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5656576,"data_stored":49015056,"data_compressed":1226019,"data_compressed_allocated":3010560,"data_compressed_original":46871818,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3829760,"data_stored":30976428,"data_compressed":739628,"data_compressed_allocated":2228224,"data_compressed_original":29737290,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:00:43.234 INFO:tasks.thrashosds.thrasher:Setting 4.12 to [5, 1] 2024-09-16T13:00:43.234 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.12', '5', '1'] 2024-09-16T13:00:43.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 4.12 5 1 2024-09-16T13:00:43.235 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54223 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.573457469000001, 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028808500000000001, 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54223 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655798+0000", 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655799+0000", 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7999999999999996e-07 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655801+0000", 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6579999999999998e-06 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655811+0000", 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5070000000000006e-06 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655816+0000", 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8960000000000002e-06 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684450+0000", 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028634057000000001 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684521+0000", 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1453999999999997e-05 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684607+0000", 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5447999999999999e-05 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54224 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1220608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.240 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.573434667000001, 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028979718000000002, 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54224 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:43.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655821+0000", 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655822+0000", 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2600000000000002e-07 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655824+0000", 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.593e-06 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655833+0000", 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2099999999999999e-06 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655838+0000", 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5650000000000003e-06 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684637+0000", 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028798550999999999 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684715+0000", 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8242000000000001e-05 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684801+0000", 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6031000000000004e-05 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54225 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.57341255, 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029147567999999999, 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54225 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655843+0000", 2024-09-16T13:00:43.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655844+0000", 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.06e-07 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655846+0000", 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5059999999999998e-06 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655856+0000", 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.825e-06 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655861+0000", 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6990000000000002e-06 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684830+0000", 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028969385 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684907+0000", 2024-09-16T13:00:43.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6800999999999999e-05 2024-09-16T13:00:43.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.684991+0000", 2024-09-16T13:00:43.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3845999999999996e-05 2024-09-16T13:00:43.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.249 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54226 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.57338961, 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029452944000000002, 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54226 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655866+0000", 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655867+0000", 2024-09-16T13:00:43.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8599999999999998e-07 2024-09-16T13:00:43.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655869+0000", 2024-09-16T13:00:43.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5849999999999998e-06 2024-09-16T13:00:43.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655878+0000", 2024-09-16T13:00:43.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2620000000000004e-06 2024-09-16T13:00:43.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655883+0000", 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7350000000000004e-06 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685017+0000", 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029134083000000002 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685096+0000", 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999994e-05 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685319+0000", 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022352299999999999 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54227 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1318912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.573367123000001, 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029655159, 2024-09-16T13:00:43.255 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54227 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655889+0000", 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5700000000000002e-07 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655892+0000", 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.486e-06 2024-09-16T13:00:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655903+0000", 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1e-05 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655907+0000", 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.831e-06 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685347+0000", 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029439844 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685416+0000", 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8561999999999998e-05 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685544+0000", 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012787900000000001 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54228 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1351680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.573342778000001, 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029867089999999999, 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54228 2024-09-16T13:00:43.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655913+0000", 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2499999999999995e-07 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655916+0000", 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9170000000000002e-06 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655926+0000", 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7769999999999997e-06 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655931+0000", 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8470000000000003e-06 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685571+0000", 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029640061999999998 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685642+0000", 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1327000000000003e-05 2024-09-16T13:00:43.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685780+0000", 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013763499999999999 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54229 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.573319071, 2024-09-16T13:00:43.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030082340999999999, 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54229 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655937+0000", 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1500000000000005e-07 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655940+0000", 2024-09-16T13:00:43.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5840000000000002e-06 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655949+0000", 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9979999999999992e-06 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.655954+0000", 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.746e-06 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685808+0000", 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029854776999999999 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.685879+0000", 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0593000000000007e-05 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.686019+0000", 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000140128 2024-09-16T13:00:43.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54832 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 192512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.526665906000002, 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.032379022, 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54832 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702590+0000", 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4099999999999999e-07 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702592+0000", 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.995e-06 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702597+0000", 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2120000000000002e-06 2024-09-16T13:00:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.702600+0000", 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9919999999999999e-06 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.727494+0000", 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024893141000000001 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.727564+0000", 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0761999999999998e-05 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734969+0000", 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0074044790000000003 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54858 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1044480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.273 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525347526000001, 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028959381999999999, 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54858 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:43.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703908+0000", 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703909+0000", 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7599999999999997e-07 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703911+0000", 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4789999999999999e-06 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703917+0000", 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2389999999999999e-06 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703920+0000", 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.021e-06 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732697+0000", 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028776665 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732778+0000", 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1018000000000003e-05 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732868+0000", 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9483999999999999e-05 2024-09-16T13:00:43.277 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54859 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1077248~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525330091000001, 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029138931999999999, 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54859 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703926+0000", 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9400000000000001e-07 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703928+0000", 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1179999999999999e-06 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703934+0000", 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.84e-06 2024-09-16T13:00:43.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703938+0000", 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5219999999999999e-06 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732893+0000", 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028955005999999998 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.732974+0000", 2024-09-16T13:00:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1569000000000005e-05 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733065+0000", 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0482999999999994e-05 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54860 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1110016~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525314011999999, 2024-09-16T13:00:43.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029318429999999999, 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54860 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703942+0000", 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.703944+0000", 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9209999999999999e-06 2024-09-16T13:00:43.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704020+0000", 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6217999999999994e-05 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704027+0000", 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3559999999999998e-06 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733091+0000", 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029064349999999999 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733167+0000", 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6058999999999998e-05 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733260+0000", 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3110999999999997e-05 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54861 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525222821, 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029431589000000001, 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54861 2024-09-16T13:00:43.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704033+0000", 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704034+0000", 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6199999999999997e-07 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704037+0000", 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9100000000000001e-06 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704045+0000", 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2150000000000001e-06 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704048+0000", 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6380000000000002e-06 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733285+0000", 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029236999999999999 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733365+0000", 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9461000000000004e-05 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733465+0000", 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9603000000000002e-05 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54862 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1175552~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525203148999999, 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029595202000000001, 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54862 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704053+0000", 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3300000000000002e-07 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704055+0000", 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0940000000000002e-06 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704061+0000", 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.835e-06 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704064+0000", 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.948e-06 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733485+0000", 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029420565999999999 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733552+0000", 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7409999999999993e-05 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733648+0000", 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5816000000000006e-05 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.295 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54863 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1208320~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525188351000001, 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029759681, 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54863 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704067+0000", 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704068+0000", 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704076+0000", 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0299999999999994e-06 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704082+0000", 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.755e-06 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704085+0000", 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1159999999999999e-06 2024-09-16T13:00:43.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733670+0000", 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029585163000000001 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733742+0000", 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2069000000000004e-05 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733827+0000", 2024-09-16T13:00:43.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5161000000000007e-05 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54864 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525167227000001, 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927993, 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54864 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:43.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704089+0000", 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3599999999999999e-07 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704091+0000", 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-06 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704096+0000", 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0950000000000003e-06 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704099+0000", 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8329999999999998e-06 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733853+0000", 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029754368999999999 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.733931+0000", 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7956999999999999e-05 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734017+0000", 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5272999999999995e-05 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54865 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1273856~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.304 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525153293999999, 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030108264999999999, 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54865 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:43.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704102+0000", 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704103+0000", 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704104+0000", 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.716e-06 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704110+0000", 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4410000000000001e-06 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704113+0000", 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1839999999999999e-06 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734040+0000", 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029927013999999998 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734122+0000", 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1481000000000001e-05 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734211+0000", 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9140000000000004e-05 2024-09-16T13:00:43.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54866 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1306624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525138883, 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030274079999999998, 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54866 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704117+0000", 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704119+0000", 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9870000000000002e-06 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704125+0000", 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3410000000000002e-06 2024-09-16T13:00:43.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704133+0000", 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734225+0000", 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030092026000000001 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734306+0000", 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0634000000000001e-05 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734391+0000", 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5082999999999998e-05 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54867 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:43.313 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525118809999999, 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030392758999999998, 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54867 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704137+0000", 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704139+0000", 2024-09-16T13:00:43.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.99e-06 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704145+0000", 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704148+0000", 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.078e-06 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734412+0000", 2024-09-16T13:00:43.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030264287000000001 2024-09-16T13:00:43.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734469+0000", 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6925999999999997e-05 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734530+0000", 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0381999999999999e-05 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.321 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54868 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1372160~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.525104115000001, 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030530140000000001, 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54868 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704152+0000", 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704245+0000", 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3037999999999999e-05 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704256+0000", 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0718e-05 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704261+0000", 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734544+0000", 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030283929000000001 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734595+0000", 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0356999999999999e-05 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734682+0000", 2024-09-16T13:00:43.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6938e-05 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54869 4.1c 4:3e277d9d:::rbd_data.11b58f0917ce.0000000000000000:1a [read 1404928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.524989874999999, 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030584762000000001, 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54869 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704266+0000", 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704267+0000", 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4700000000000001e-07 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704269+0000", 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1390000000000002e-06 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704275+0000", 2024-09-16T13:00:43.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9449999999999998e-06 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.704278+0000", 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1429999999999998e-06 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734705+0000", 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030427551000000001 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734778+0000", 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2502000000000005e-05 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.734851+0000", 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2835000000000003e-05 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.330 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:43.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:43.333 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:43.324+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:43.421 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T13:00:43.433 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:43.425+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:43.453 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:43.453 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:43.453 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:43.453 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:43.464 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:43.534 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:43.528+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:43.615 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:43.615 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:43.615 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:43.616 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:43.616 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:43.626 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:43.635 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:43.629+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:43.638 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-16T13:00:43.736 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:43.728+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:43.806 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.731957381000001, 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:00:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:00:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:00:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.731904962000002, 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:00:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.731888114, 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.731871999999999, 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.731807069999999, 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:00:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:00:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:00:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.731790970999999, 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:00:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:00:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:00:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:00:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.731717190000001, 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:00:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:00:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.731699988999999, 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:00:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:00:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:00:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.731683285999999, 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:00:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.195372031, 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:00:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:00:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:00:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.195354894999999, 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:00:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.195335312999999, 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:00:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.195304454, 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.195236358000001, 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:00:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159502 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4493312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612957+0000", 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.187948668000001, 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070340508999999996, 2024-09-16T13:00:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159502 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612957+0000", 2024-09-16T13:00:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612957+0000", 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612958+0000", 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612960+0000", 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3750000000000001e-06 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612967+0000", 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3929999999999996e-06 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612970+0000", 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.23e-06 2024-09-16T13:00:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683111+0000", 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070141677999999999 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683188+0000", 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6157999999999998e-05 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683298+0000", 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110221 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.187932503000001, 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:00:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:00:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.187914914, 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:00:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:00:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:00:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:00:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.187900021000001, 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:00:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.187868404, 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:00:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.187850383000001, 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:00:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:43.903 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:43.904 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:43.905 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:43.897+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:44.005 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:43.997+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:44.009 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:44.009 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:44.009 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:44.009 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:44.019 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:44.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:44.108 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:44.100+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:44.123 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:44.123 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:44.123 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:44.124 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:44.124 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:44.135 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:44.207 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:44.201+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:44.226 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240663 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 145408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543034+0000", 2024-09-16T13:00:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6791552549999995, 2024-09-16T13:00:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047682317000000002, 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240663 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543034+0000", 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543034+0000", 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543034+0000", 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1299999999999999e-07 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543042+0000", 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0320000000000003e-06 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543050+0000", 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543054+0000", 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6720000000000003e-06 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590615+0000", 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047560534000000002 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590664+0000", 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8903000000000003e-05 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590716+0000", 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2243000000000001e-05 2024-09-16T13:00:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240664 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 152576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.679128006, 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047790817999999999, 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240664 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1900000000000003e-07 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543064+0000", 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4449999999999999e-06 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543079+0000", 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5221e-05 2024-09-16T13:00:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543084+0000", 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8590000000000004e-06 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590750+0000", 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047665710999999999 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590801+0000", 2024-09-16T13:00:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1827999999999999e-05 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590852+0000", 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0235000000000003e-05 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240665 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:00:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6790986029999999, 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047896192999999997, 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240665 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543091+0000", 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543094+0000", 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7149999999999998e-06 2024-09-16T13:00:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543102+0000", 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6549999999999993e-06 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543108+0000", 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4840000000000003e-06 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590885+0000", 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047777396 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590936+0000", 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0611e-05 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590987+0000", 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0785000000000003e-05 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240666 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 162816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6790763349999995, 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048010069000000002, 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240666 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8500000000000002e-07 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543116+0000", 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.531e-06 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543123+0000", 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1069999999999999e-06 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543127+0000", 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4309999999999996e-06 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591020+0000", 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047892413000000002 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591071+0000", 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1650000000000002e-05 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591123+0000", 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1452000000000001e-05 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240667 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6790575529999998, 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048125951, 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240667 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:00:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543132+0000", 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5099999999999999e-07 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543134+0000", 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1500000000000002e-06 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543141+0000", 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543146+0000", 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9200000000000003e-06 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591158+0000", 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048011246 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591207+0000", 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9295999999999997e-05 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591257+0000", 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0426000000000001e-05 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240668 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 173056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6790378480000001, 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048243280999999999, 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240668 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543152+0000", 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543154+0000", 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5380000000000001e-06 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543161+0000", 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7920000000000004e-06 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543172+0000", 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1327e-05 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591290+0000", 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048118181000000003 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591344+0000", 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4061000000000003e-05 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591394+0000", 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9866e-05 2024-09-16T13:00:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240766 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:00:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6778335230000003, 2024-09-16T13:00:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047716644000000003, 2024-09-16T13:00:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240766 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544356+0000", 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2500000000000005e-07 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544373+0000", 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6685999999999999e-05 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544381+0000", 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7190000000000001e-06 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544384+0000", 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591969+0000", 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047585483999999997 2024-09-16T13:00:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592021+0000", 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1839000000000003e-05 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592072+0000", 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754000000000002e-05 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240767 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 201728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.677801648, 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047822479000000001, 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240767 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544388+0000", 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.51e-07 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544390+0000", 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.824e-06 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544393+0000", 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.681e-06 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544396+0000", 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4940000000000002e-06 2024-09-16T13:00:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592105+0000", 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047709184000000002 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592158+0000", 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2772999999999998e-05 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592210+0000", 2024-09-16T13:00:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2071999999999999e-05 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240768 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6777888470000004, 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047944647, 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240768 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0400000000000002e-07 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544403+0000", 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1430000000000001e-06 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544406+0000", 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7450000000000001e-06 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544409+0000", 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592241+0000", 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047832146999999998 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592291+0000", 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9289000000000002e-05 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592345+0000", 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4067000000000003e-05 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240769 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6777772219999996, 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048066225999999997, 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240769 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1399999999999998e-07 2024-09-16T13:00:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544413+0000", 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.42e-06 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544417+0000", 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.529e-06 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544420+0000", 2024-09-16T13:00:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5620000000000002e-06 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592378+0000", 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047958354000000002 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592427+0000", 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9013e-05 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592478+0000", 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1034e-05 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240770 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6777669589999995, 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048189432999999997, 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240770 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3200000000000001e-07 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544424+0000", 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4449999999999999e-06 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544427+0000", 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5049999999999999e-06 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544430+0000", 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.61e-06 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592511+0000", 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048080912000000003 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592563+0000", 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2107000000000001e-05 2024-09-16T13:00:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592611+0000", 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8522000000000001e-05 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240771 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6777323519999996, 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290128000000002, 2024-09-16T13:00:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240771 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5700000000000002e-07 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544459+0000", 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6840000000000001e-06 2024-09-16T13:00:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544466+0000", 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999997e-06 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544469+0000", 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592645+0000", 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048175216999999999 2024-09-16T13:00:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592696+0000", 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1292999999999998e-05 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592747+0000", 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0751000000000002e-05 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240772 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6777090569999995, 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048403333999999999, 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240772 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544482+0000", 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8390000000000001e-06 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544486+0000", 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9369999999999997e-06 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544489+0000", 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8459999999999999e-06 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592779+0000", 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290301000000001 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592831+0000", 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2335e-05 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592883+0000", 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1786999999999997e-05 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240773 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6776976719999999, 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048527963, 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240773 2024-09-16T13:00:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:00:44.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544492+0000", 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.84e-07 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544493+0000", 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6279999999999999e-06 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544498+0000", 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4689999999999999e-06 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544500+0000", 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592916+0000", 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048415672 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592965+0000", 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9120999999999999e-05 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593019+0000", 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3950999999999999e-05 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240774 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6776860170000001, 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048649546000000002, 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240774 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:00:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544505+0000", 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.61e-06 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544508+0000", 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4929999999999998e-06 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544511+0000", 2024-09-16T13:00:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4169999999999999e-06 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593051+0000", 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048539604 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593103+0000", 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2068999999999999e-05 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593152+0000", 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9870999999999997e-05 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240775 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6776757629999999, 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048773328999999997, 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240775 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:00:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545618+0000", 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011045810000000001 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545625+0000", 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6079999999999998e-06 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545644+0000", 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.823e-05 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545653+0000", 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3000000000000007e-06 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593186+0000", 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047533387000000003 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593235+0000", 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8767000000000003e-05 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593286+0000", 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1456000000000003e-05 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240776 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6765236249999997, 2024-09-16T13:00:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047754889000000002, 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240776 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545666+0000", 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-06 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545671+0000", 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2270000000000004e-06 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545681+0000", 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0506000000000001e-05 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545687+0000", 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0970000000000002e-06 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593320+0000", 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047632999000000002 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593369+0000", 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8882999999999999e-05 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593420+0000", 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1127e-05 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240777 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:00:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6764950039999995, 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047878318000000003, 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240777 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545695+0000", 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545698+0000", 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0709999999999999e-06 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545707+0000", 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7629999999999995e-06 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545713+0000", 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2839999999999997e-06 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593453+0000", 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047740281000000002 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593518+0000", 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5493000000000005e-05 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593572+0000", 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3739000000000002e-05 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240778 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6764705649999998, 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047999411999999998, 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240778 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545719+0000", 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.13e-07 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545728+0000", 2024-09-16T13:00:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1560000000000001e-06 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545901+0000", 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017311599999999999 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545909+0000", 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.028e-06 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593606+0000", 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047696660000000002 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593666+0000", 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9611999999999998e-05 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593718+0000", 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2227e-05 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240951 4.8 4:10cc0d5f:::rbd_data.11b552946c0f.0000000000000120:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6683902960000001, 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048226894999999999, 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240951 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2300000000000002e-07 2024-09-16T13:00:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553801+0000", 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.686e-06 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553805+0000", 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3039999999999998e-06 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553809+0000", 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.599994+0000", 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046185154999999999 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.600112+0000", 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118589 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.602026+0000", 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019131370000000001 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:44.317 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:44.317 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:44.318 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:44.321 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:44.312+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:44.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:44.397 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:44.424 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:44.412+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:44.439 INFO:teuthology.orchestra.run.smithi042.stderr:set 4.12 pg_upmap mapping to [5,1] 2024-09-16T13:00:44.440 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:44.444 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:44.445 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:44.455 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:44.461 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T13:00:44.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:44.470 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:44.473 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:44.473 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:44.520 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:44.513+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:44.547 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:44.547 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:44.557 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:44.621 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:44.612+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:44.653 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:44.653 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:44.662 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:44.721 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:44.713+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:44.769 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:44.770 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:44.779 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:44.822 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:44.817+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:44.876 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:44.876 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:44.885 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:44.923 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:44.916+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:44.982 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:44.983 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:44.992 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:45.029 INFO:teuthology.orchestra.run.smithi042.stdout:5328 write 0x4a26be3 thru 0x4a35943 (0xed61 bytes) 2024-09-16T13:00:45.030 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:45.024+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:45.045 INFO:teuthology.orchestra.run.smithi042.stdout:5329 write 0x393d33e thru 0x39470fc (0x9dbf bytes) 2024-09-16T13:00:45.059 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:45.061 INFO:teuthology.orchestra.run.smithi042.stdout:5331 trunc from 0x82cb242 to 0x6f2e94d 2024-09-16T13:00:45.089 INFO:teuthology.orchestra.run.smithi042.stdout:5332 read 0x567f828 thru 0x568449e (0x4c77 bytes) 2024-09-16T13:00:45.090 INFO:teuthology.orchestra.run.smithi042.stdout:5334 punch from 0x48852aa to 0x4885a73, (0x7c9 bytes) 2024-09-16T13:00:45.091 INFO:teuthology.orchestra.run.smithi042.stdout:5335 punch from 0x12bfdc to 0x13ae6a, (0xee8e bytes) 2024-09-16T13:00:45.091 INFO:teuthology.orchestra.run.smithi042.stdout:5337 punch from 0xeb51ff to 0xec2aff, (0xd900 bytes) 2024-09-16T13:00:45.100 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:45.100 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:45.106 INFO:teuthology.orchestra.run.smithi042.stdout:5338 writesame 0xa4d7348 thru 0xa4e3021 data_size 0x17(0xbcda bytes) 2024-09-16T13:00:45.106 INFO:teuthology.orchestra.run.smithi042.stdout:5339 writesame 0x6637fa7 thru 0x6640280 data_size 0x9(0x82da bytes) 2024-09-16T13:00:45.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:45.122 INFO:teuthology.orchestra.run.smithi042.stdout:5340 write 0xb95cc9f thru 0xb965ae7 (0x8e49 bytes) 2024-09-16T13:00:45.126 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:45.117+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:45.128 INFO:teuthology.orchestra.run.smithi042.stdout:5342 writesame 0x2f68b55 thru 0x2f73675 data_size 0x11(0xab21 bytes) 2024-09-16T13:00:45.141 INFO:teuthology.orchestra.run.smithi042.stdout:5343 read 0x6d86d81 thru 0x6d96717 (0xf997 bytes) 2024-09-16T13:00:45.161 INFO:teuthology.orchestra.run.smithi042.stdout:5344 trunc from 0xb965ae8 to 0x2aebea 2024-09-16T13:00:45.208 INFO:teuthology.orchestra.run.smithi042.stdout:5345 write 0x1f12207 thru 0x1f1c0a0 (0x9e9a bytes) 2024-09-16T13:00:45.215 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:45.215 INFO:teuthology.orchestra.run.smithi042.stdout:5348 read 0xcb21dd thru 0xcb9f4c (0x7d70 bytes) 2024-09-16T13:00:45.215 INFO:teuthology.orchestra.run.smithi042.stdout:5350 read 0x19720c9 thru 0x197a3b9 (0x82f1 bytes) 2024-09-16T13:00:45.215 INFO:teuthology.orchestra.run.smithi042.stdout:5352 read 0x1acabb2 thru 0x1ace68c (0x3adb bytes) 2024-09-16T13:00:45.216 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:45.226 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:45.221+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:45.234 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:45.234 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:45.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:45.247 INFO:teuthology.orchestra.run.smithi042.stdout:5357 write 0xee6277c thru 0xee6513c (0x29c1 bytes) 2024-09-16T13:00:45.254 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:45.267 INFO:teuthology.orchestra.run.smithi042.stdout:5360 trunc from 0xee6513d to 0x52d002c 2024-09-16T13:00:45.289 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:45.289 INFO:teuthology.orchestra.run.smithi042.stdout:5363 punch from 0x162700e to 0x16327fc, (0xb7ee bytes) 2024-09-16T13:00:45.300 INFO:teuthology.orchestra.run.smithi042.stdout:5364 trunc from 0x52d002c to 0xbc3bf51 2024-09-16T13:00:45.308 INFO:teuthology.orchestra.run.smithi042.stdout:5365 punch from 0xa09fb62 to 0xa0a9835, (0x9cd3 bytes) 2024-09-16T13:00:45.308 INFO:teuthology.orchestra.run.smithi042.stdout:5371 writesame 0x9f34be1 thru 0x9f42583 data_size 0x5(0xd9a3 bytes) 2024-09-16T13:00:45.322 INFO:teuthology.orchestra.run.smithi042.stdout:5377 punch from 0x8d3fadd to 0x8d4ae1d, (0xb340 bytes) 2024-09-16T13:00:45.322 INFO:teuthology.orchestra.run.smithi042.stdout:5379 write 0x4418aca thru 0x4423f07 (0xb43e bytes) 2024-09-16T13:00:45.326 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:45.320+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:45.336 INFO:teuthology.orchestra.run.smithi042.stdout:5380 trunc from 0xbc3bf51 to 0x7e24914 2024-09-16T13:00:45.345 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:45.345 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:45.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:45.355 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:45.368 INFO:teuthology.orchestra.run.smithi042.stdout:5382 write 0xaaebd52 thru 0xaaf2572 (0x6821 bytes) 2024-09-16T13:00:45.374 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:45.374 INFO:teuthology.orchestra.run.smithi042.stdout:5384 read 0x6683ee2 thru 0x668afbb (0x70da bytes) 2024-09-16T13:00:45.374 INFO:teuthology.orchestra.run.smithi042.stdout:5385 punch from 0x6c92437 to 0x6c95486, (0x304f bytes) 2024-09-16T13:00:45.374 INFO:teuthology.orchestra.run.smithi042.stdout:5386 read 0x3e84734 thru 0x3e8bb8e (0x745b bytes) 2024-09-16T13:00:45.374 INFO:teuthology.orchestra.run.smithi042.stdout:5387 write 0x8cf739f thru 0x8cfa224 (0x2e86 bytes) 2024-09-16T13:00:45.381 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:45.382 INFO:teuthology.orchestra.run.smithi042.stdout:5391 write 0x499b215 thru 0x499f8cd (0x46b9 bytes) 2024-09-16T13:00:45.387 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:45.388 INFO:teuthology.orchestra.run.smithi042.stdout:5395 read 0x2acedb2 thru 0x2ad054b (0x179a bytes) 2024-09-16T13:00:45.405 INFO:teuthology.orchestra.run.smithi042.stdout:5396 write 0xed542e1 thru 0xed61113 (0xce33 bytes) 2024-09-16T13:00:45.412 INFO:teuthology.orchestra.run.smithi042.stdout:5397 write 0xd992079 thru 0xd996f41 (0x4ec9 bytes) 2024-09-16T13:00:45.419 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:45.419 INFO:teuthology.orchestra.run.smithi042.stdout:5399 read 0x6d53590 thru 0x6d5f503 (0xbf74 bytes) 2024-09-16T13:00:45.419 INFO:teuthology.orchestra.run.smithi042.stdout:5402 punch from 0x411617 to 0x41eef6, (0xd8df bytes) 2024-09-16T13:00:45.419 INFO:teuthology.orchestra.run.smithi042.stdout:5404 punch from 0x3f3cbe4 to 0x3f435a1, (0x69bd bytes) 2024-09-16T13:00:45.420 INFO:teuthology.orchestra.run.smithi042.stdout:5406 punch from 0x745cd60 to 0x74668af, (0x9b4f bytes) 2024-09-16T13:00:45.420 INFO:teuthology.orchestra.run.smithi042.stdout:5407 write 0x248e927 thru 0x249ce8f (0xe569 bytes) 2024-09-16T13:00:45.427 INFO:teuthology.orchestra.run.smithi042.stdout:5408 write 0x8e24943 thru 0x8e2d4d5 (0x8b93 bytes) 2024-09-16T13:00:45.428 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:45.420+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:45.442 INFO:teuthology.orchestra.run.smithi042.stdout:5409 write 0x662198b thru 0x662415c (0x27d2 bytes) 2024-09-16T13:00:45.446 INFO:teuthology.orchestra.run.smithi042.stdout:5410 write 0x3123d05 thru 0x312a2c0 (0x65bc bytes) 2024-09-16T13:00:45.451 INFO:teuthology.orchestra.run.smithi042.stdout:5411 write 0x58a3928 thru 0x58b2e26 (0xf4ff bytes) 2024-09-16T13:00:45.456 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:45.456 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:45.468 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:45.471 INFO:teuthology.orchestra.run.smithi042.stdout:5412 trunc from 0xed61114 to 0x4f16fae 2024-09-16T13:00:45.494 INFO:teuthology.orchestra.run.smithi042.stdout:5413 write 0x40a7236 thru 0x40b2b56 (0xb921 bytes) 2024-09-16T13:00:45.510 INFO:teuthology.orchestra.run.smithi042.stdout:5414 write 0x787492e thru 0x78753f8 (0xacb bytes) 2024-09-16T13:00:45.517 INFO:teuthology.orchestra.run.smithi042.stdout:5415 trunc from 0x78753f9 to 0xad57b25 2024-09-16T13:00:45.527 INFO:teuthology.orchestra.run.smithi042.stdout:5416 read 0x21b0612 thru 0x21b376d (0x315c bytes) 2024-09-16T13:00:45.527 INFO:teuthology.orchestra.run.smithi042.stdout:5418 read 0x8949b6b thru 0x8958783 (0xec19 bytes) 2024-09-16T13:00:45.528 INFO:teuthology.orchestra.run.smithi042.stdout:5420 punch from 0x7718e6d to 0x77243ee, (0xb581 bytes) 2024-09-16T13:00:45.528 INFO:teuthology.orchestra.run.smithi042.stdout:5421 punch from 0x6a1963a to 0x6a1e254, (0x4c1a bytes) 2024-09-16T13:00:45.528 INFO:teuthology.orchestra.run.smithi042.stdout:5422 clone 35 order 19 su 262144 sc 9 2024-09-16T13:00:45.529 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:45.521+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:45.571 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:45.571 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:45.579 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:45.630 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:45.621+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:45.720 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:45.721 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:45.729 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:45.730 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:45.725+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:45.831 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:45.831 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:45.832 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:45.824+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:45.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:45.933 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:45.928+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:45.940 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:45.940 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:45.949 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:46.034 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:46.028+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:46.048 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:46.048 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:46.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:46.135 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:46.129+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:46.199 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:46.208 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:46.236 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:46.228+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:46.309 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:46.309 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:46.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:46.357 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:46.352+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:46.438 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:46.432+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:46.463 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T13:00:46.472 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:46.472 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:46.474 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:46.483 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:46.538 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone34 from 0xad57b25 (overlap 0x2aebea) to 0x0 2024-09-16T13:00:46.538 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:46.532+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:46.600 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:46.600 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:46.610 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:46.611 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:46.613 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-16T13:00:46.633 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:46.639 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:46.632+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:46.654 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:46.677 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:46.711 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:46.711 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:46.721 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:46.740 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:46.732+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:46.824 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:46.824 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:46.833 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:46.845 INFO:teuthology.orchestra.run.smithi042.stdout:5423 read 0x3a86b38 thru 0x3a8a687 (0x3b50 bytes) 2024-09-16T13:00:46.847 INFO:teuthology.orchestra.run.smithi042.stdout:5424 read 0x3a6555f thru 0x3a716b6 (0xc158 bytes) 2024-09-16T13:00:46.849 INFO:teuthology.orchestra.run.smithi042.stdout:5425 punch from 0x25d346f to 0x25d584c, (0x23dd bytes) 2024-09-16T13:00:46.860 INFO:teuthology.orchestra.run.smithi042.stdout:5427 trunc from 0xad57b25 to 0x7df7244 2024-09-16T13:00:46.879 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:46.873+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:46.880 INFO:teuthology.orchestra.run.smithi042.stdout:5430 read 0x75b186 thru 0x75fdfd (0x4c78 bytes) 2024-09-16T13:00:46.885 INFO:teuthology.orchestra.run.smithi042.stdout:5431 trunc from 0x7df7244 to 0x9f61bd4 2024-09-16T13:00:46.897 INFO:teuthology.orchestra.run.smithi042.stdout:5432 read 0x1187d9c thru 0x11969a9 (0xec0e bytes) 2024-09-16T13:00:46.898 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:46.898 INFO:teuthology.orchestra.run.smithi042.stdout:5434 punch from 0x6c18b01 to 0x6c1c4ec, (0x39eb bytes) 2024-09-16T13:00:46.898 INFO:teuthology.orchestra.run.smithi042.stdout:5435 read 0x22ed658 thru 0x22f748e (0x9e37 bytes) 2024-09-16T13:00:46.912 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:46.917 INFO:teuthology.orchestra.run.smithi042.stdout:5437 write 0xe359ef6 thru 0xe366583 (0xc68e bytes) 2024-09-16T13:00:46.923 INFO:teuthology.orchestra.run.smithi042.stdout:5439 read 0x77b006c thru 0x77b2b7c (0x2b11 bytes) 2024-09-16T13:00:46.923 INFO:teuthology.orchestra.run.smithi042.stdout:5440 writesame 0x4a7302c thru 0x4a7b918 data_size 0x1(0x88ed bytes) 2024-09-16T13:00:46.935 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:46.935 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:46.941 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:46.941 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:46.936+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:46.944 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:46.952 INFO:teuthology.orchestra.run.smithi042.stdout:5442 read 0xd2f861f thru 0xd301af9 (0x94db bytes) 2024-09-16T13:00:46.960 INFO:teuthology.orchestra.run.smithi042.stdout:5443 write 0xe3eb5bf thru 0xe3f6279 (0xacbb bytes) 2024-09-16T13:00:46.965 INFO:teuthology.orchestra.run.smithi042.stdout:5444 punch from 0xa8d27d4 to 0xa8da81a, (0x8046 bytes) 2024-09-16T13:00:46.965 INFO:teuthology.orchestra.run.smithi042.stdout:5445 write 0xcb2921d thru 0xcb2c589 (0x336d bytes) 2024-09-16T13:00:46.969 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:46.969 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:46.970 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:46.970 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:46.970 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:46.970 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:46.970 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:46.970 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:46.970 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:46.970 INFO:teuthology.orchestra.run.smithi042.stdout:5446 write 0xaafe38f thru 0xab056ab (0x731d bytes) 2024-09-16T13:00:46.972 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:46.972 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:46.978 INFO:teuthology.orchestra.run.smithi042.stdout:5448 read 0x455ef44 thru 0x456a104 (0xb1c1 bytes) 2024-09-16T13:00:46.978 INFO:teuthology.orchestra.run.smithi042.stdout:5449 read 0x412d6c5 thru 0x4137021 (0x995d bytes) 2024-09-16T13:00:46.979 INFO:teuthology.orchestra.run.smithi042.stdout:5451 writesame 0x23b7a4f thru 0x23c53e4 data_size 0x1(0xd996 bytes) 2024-09-16T13:00:47.011 INFO:teuthology.orchestra.run.smithi042.stdout:5453 read 0xb61cd7d thru 0xb61ddc6 (0x104a bytes) 2024-09-16T13:00:47.011 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:47.020 INFO:teuthology.orchestra.run.smithi042.stdout:5457 write 0xecc9e21 thru 0xecd2b75 (0x8d55 bytes) 2024-09-16T13:00:47.026 INFO:teuthology.orchestra.run.smithi042.stdout:5458 write 0xe1a3b4c thru 0xe1a7dac (0x4261 bytes) 2024-09-16T13:00:47.029 INFO:teuthology.orchestra.run.smithi042.stdout:5459 punch from 0x102f2bd to 0x1034d51, (0x5a94 bytes) 2024-09-16T13:00:47.029 INFO:teuthology.orchestra.run.smithi042.stdout:5460 read 0x5a3d696 thru 0x5a42b30 (0x549b bytes) 2024-09-16T13:00:47.030 INFO:teuthology.orchestra.run.smithi042.stdout:5461 writesame 0xc645202 thru 0xc6482d6 data_size 0x9(0x30d5 bytes) 2024-09-16T13:00:47.035 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:47.035 INFO:teuthology.orchestra.run.smithi042.stdout:5466 read 0xa0715ef thru 0xa071a59 (0x46b bytes) 2024-09-16T13:00:47.035 INFO:teuthology.orchestra.run.smithi042.stdout:5468 write 0xbcb95b6 thru 0xbcc7058 (0xdaa3 bytes) 2024-09-16T13:00:47.042 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:47.042 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:47.042 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:47.042 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:47.042 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:47.043 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:47.036+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:47.044 INFO:teuthology.orchestra.run.smithi042.stdout:5472 write 0x3511cb1 thru 0x351b3a5 (0x96f5 bytes) 2024-09-16T13:00:47.051 INFO:teuthology.orchestra.run.smithi042.stdout:5473 read 0xa6aa680 thru 0xa6ab73d (0x10be bytes) 2024-09-16T13:00:47.051 INFO:teuthology.orchestra.run.smithi042.stdout:5474 write 0x182acfe thru 0x18394ce (0xe7d1 bytes) 2024-09-16T13:00:47.052 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:47.059 INFO:teuthology.orchestra.run.smithi042.stdout:5475 write 0x52f5fff thru 0x5300d19 (0xad1b bytes) 2024-09-16T13:00:47.069 INFO:teuthology.orchestra.run.smithi042.stdout:5477 write 0xc952b6c thru 0xc96087c (0xdd11 bytes) 2024-09-16T13:00:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:5479 read 0xa9dda27 thru 0xa9eb39b (0xd975 bytes) 2024-09-16T13:00:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:5481 read 0x6c2de7 thru 0x6cc478 (0x9692 bytes) 2024-09-16T13:00:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:5483 read 0x6da13c7 thru 0x6da5d2f (0x4969 bytes) 2024-09-16T13:00:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:5484 read 0x79402ce thru 0x79466b9 (0x63ec bytes) 2024-09-16T13:00:47.076 INFO:teuthology.orchestra.run.smithi042.stdout:5485 read 0x70555d8 thru 0x7062bd3 (0xd5fc bytes) 2024-09-16T13:00:47.077 INFO:teuthology.orchestra.run.smithi042.stdout:5486 read 0x5b19817 thru 0x5b22451 (0x8c3b bytes) 2024-09-16T13:00:47.077 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:47.077 INFO:teuthology.orchestra.run.smithi042.stdout:5488 write 0x2a466dc thru 0x2a5461f (0xdf44 bytes) 2024-09-16T13:00:47.084 INFO:teuthology.orchestra.run.smithi042.stdout:5489 write 0xa872a05 thru 0xa881bb9 (0xf1b5 bytes) 2024-09-16T13:00:47.091 INFO:teuthology.orchestra.run.smithi042.stdout:5490 punch from 0xcea423e to 0xceafee4, (0xbca6 bytes) 2024-09-16T13:00:47.117 INFO:teuthology.orchestra.run.smithi042.stdout:5491 trunc from 0xecd2b76 to 0x26a972b 2024-09-16T13:00:47.144 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:47.136+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:47.151 INFO:teuthology.orchestra.run.smithi042.stdout:5495 punch from 0x22a286c to 0x22af67c, (0xce10 bytes) 2024-09-16T13:00:47.160 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:47.160 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:47.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:47.160 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.792606711999994, 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T13:00:47.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:18031 4.4 4:2649aabf:::rbd_data.11b51a3eb68b.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~5304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:47.165 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.996283478999999, 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94984657100000003, 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 18031 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:47.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160519+0000", 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3029999999999998e-06 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160536+0000", 2024-09-16T13:00:47.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7181999999999999e-05 2024-09-16T13:00:47.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160564+0000", 2024-09-16T13:00:47.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971e-05 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160576+0000", 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160669+0000", 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3282999999999994e-05 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160680+0000", 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107188+0000", 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94650794999999999 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107644+0000", 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00045604499999999997 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.108257+0000", 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061324700000000003 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.109811+0000", 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015538749999999999 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110175+0000", 2024-09-16T13:00:47.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036419399999999998 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110285+0000", 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010938900000000001 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110364+0000", 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8689000000000005e-05 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:64609 4.e 4:7394d9c8:::rbd_object_map.11b5c2ee3542:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.895074314999999, 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.020058463, 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 64609 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.173 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261729+0000", 2024-09-16T13:00:47.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4430000000000002e-06 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261741+0000", 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261780+0000", 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8722e-05 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261798+0000", 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8150000000000001e-05 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261927+0000", 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000129243 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261948+0000", 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1047e-05 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273712+0000", 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011763704999999999 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273720+0000", 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4309999999999994e-06 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280059+0000", 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.006338744 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280297+0000", 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000237667 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281645+0000", 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001348555 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281702+0000", 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6342999999999999e-05 2024-09-16T13:00:47.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281785+0000", 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2782999999999997e-05 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254540 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 176128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816406+0000", 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3403940909999998, 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053609236999999997, 2024-09-16T13:00:47.179 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254540 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816406+0000", 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816406+0000", 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816407+0000", 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6000000000000004e-07 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816409+0000", 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1349999999999999e-06 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816415+0000", 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9780000000000002e-06 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816424+0000", 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0990000000000005e-06 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.869803+0000", 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053378429999999998 2024-09-16T13:00:47.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.869904+0000", 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101937 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870016+0000", 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111098 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254541 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 183296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816431+0000", 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3403690570000002, 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053814215999999998, 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254541 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816431+0000", 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816431+0000", 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816432+0000", 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:00:47.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816441+0000", 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8550000000000008e-06 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816457+0000", 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5509e-05 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816463+0000", 2024-09-16T13:00:47.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8800000000000002e-06 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870060+0000", 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053596442000000001 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870142+0000", 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2232999999999997e-05 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870246+0000", 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103509 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254542 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 186368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816469+0000", 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.340331017, 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054024784999999999, 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.188 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254542 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816469+0000", 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816469+0000", 2024-09-16T13:00:47.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816470+0000", 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1999999999999998e-07 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816472+0000", 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.255e-06 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816479+0000", 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2539999999999998e-06 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816484+0000", 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.062e-06 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870290+0000", 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053806132 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870379+0000", 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9128000000000005e-05 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870494+0000", 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115134 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254543 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 193536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.192 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3403126379999999, 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054211546999999999, 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254543 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2e-07 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816490+0000", 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1220000000000002e-06 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816510+0000", 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9457999999999999e-05 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816522+0000", 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2585000000000001e-05 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870529+0000", 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054006981000000003 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870600+0000", 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0073999999999993e-05 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870699+0000", 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9807000000000002e-05 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.196 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254544 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 196608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816527+0000", 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3402734569999999, 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054393364999999999, 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.201 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254544 2024-09-16T13:00:47.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.201 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816527+0000", 2024-09-16T13:00:47.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816527+0000", 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816528+0000", 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-07 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816530+0000", 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1619999999999998e-06 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816542+0000", 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2459e-05 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816555+0000", 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2532e-05 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870738+0000", 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054183505 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870818+0000", 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9425999999999995e-05 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870920+0000", 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010262700000000001 2024-09-16T13:00:47.204 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254545 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 203776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816559+0000", 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.340240954, 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054590611999999997, 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254545 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816559+0000", 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816559+0000", 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816560+0000", 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9699999999999996e-07 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816562+0000", 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3549999999999999e-06 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816569+0000", 2024-09-16T13:00:47.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3879999999999997e-06 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816573+0000", 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0720000000000001e-06 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870958+0000", 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054385019 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871029+0000", 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1291000000000006e-05 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871150+0000", 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012089 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254546 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 206848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.209 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816577+0000", 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3402234599999998, 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054799678999999997, 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254546 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816577+0000", 2024-09-16T13:00:47.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816577+0000", 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816578+0000", 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6499999999999999e-07 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816580+0000", 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1270000000000001e-06 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816591+0000", 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1838000000000001e-05 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816595+0000", 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9349999999999996e-06 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871208+0000", 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054612978 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871275+0000", 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6774999999999997e-05 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871377+0000", 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101461 2024-09-16T13:00:47.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254547 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 214016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816600+0000", 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3402002230000001, 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054976553999999997, 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254547 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816600+0000", 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816600+0000", 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816601+0000", 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7000000000000005e-07 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816603+0000", 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2500000000000001e-06 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816609+0000", 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9379999999999997e-06 2024-09-16T13:00:47.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816615+0000", 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3860000000000004e-06 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871395+0000", 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054779732999999997 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871446+0000", 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0642000000000001e-05 2024-09-16T13:00:47.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871577+0000", 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013103499999999999 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254548 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 217088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816619+0000", 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.340180959, 2024-09-16T13:00:47.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055196303000000002, 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254548 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816619+0000", 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816619+0000", 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816620+0000", 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2799999999999996e-07 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816622+0000", 2024-09-16T13:00:47.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9999999999999999e-06 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816637+0000", 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4922e-05 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816641+0000", 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1269999999999996e-06 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871623+0000", 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054981587999999998 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871688+0000", 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5148999999999996e-05 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871816+0000", 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012798900000000001 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.222 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254549 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 224256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3401248900000002, 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055369889999999998, 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.223 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254549 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7499999999999999e-07 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816679+0000", 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6120000000000001e-06 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816686+0000", 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:00:47.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816690+0000", 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0439999999999997e-06 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871868+0000", 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055177858000000003 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871933+0000", 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4857000000000006e-05 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872045+0000", 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112858 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254550 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 227328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:00:47.227 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3401066529999999, 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055577646000000001, 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254550 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5599999999999995e-07 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816696+0000", 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9439999999999999e-06 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816702+0000", 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7899999999999996e-06 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816707+0000", 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0520000000000001e-06 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872108+0000", 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055400479000000002 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872184+0000", 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6123000000000003e-05 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872271+0000", 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7701999999999995e-05 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254551 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 234496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816711+0000", 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3400893630000001, 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055772654999999997, 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254551 2024-09-16T13:00:47.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816711+0000", 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816711+0000", 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816712+0000", 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2e-07 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816714+0000", 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0770000000000002e-06 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816720+0000", 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0750000000000003e-06 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816725+0000", 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0069999999999997e-06 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872306+0000", 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055580967000000002 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872386+0000", 2024-09-16T13:00:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9956999999999994e-05 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872484+0000", 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8052000000000002e-05 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254552 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 237568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816729+0000", 2024-09-16T13:00:47.236 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3400715879999998, 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055953530000000001, 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254552 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816729+0000", 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816729+0000", 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816730+0000", 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0999999999999997e-07 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816732+0000", 2024-09-16T13:00:47.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9769999999999999e-06 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816738+0000", 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0650000000000004e-06 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816743+0000", 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9969999999999998e-06 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872525+0000", 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055781981000000001 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872574+0000", 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9178000000000002e-05 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872682+0000", 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108522 2024-09-16T13:00:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254553 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3400538040000001, 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056166305999999999, 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.241 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254553 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2e-07 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816754+0000", 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9820000000000002e-06 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816760+0000", 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2024-09-16T13:00:47.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816764+0000", 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8940000000000003e-06 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872739+0000", 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055974278000000002 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872798+0000", 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9564000000000001e-05 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872913+0000", 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114825 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254554 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 247808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816768+0000", 2024-09-16T13:00:47.245 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.340032179, 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056329235999999998, 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254554 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816768+0000", 2024-09-16T13:00:47.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816768+0000", 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816769+0000", 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-07 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816771+0000", 2024-09-16T13:00:47.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.928e-06 2024-09-16T13:00:47.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816777+0000", 2024-09-16T13:00:47.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8819999999999998e-06 2024-09-16T13:00:47.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816783+0000", 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4819999999999999e-06 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872935+0000", 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056152226 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873003+0000", 2024-09-16T13:00:47.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7581000000000002e-05 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873097+0000", 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4482999999999996e-05 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254555 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 254976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.250 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3400006499999999, 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056501814999999997, 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254555 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8699999999999995e-07 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816803+0000", 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.255e-06 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816809+0000", 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3119999999999999e-06 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816813+0000", 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.856e-06 2024-09-16T13:00:47.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873130+0000", 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056317045000000003 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873192+0000", 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1857000000000001e-05 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873302+0000", 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010990299999999999 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254556 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 258048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3399836359999999, 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056686382, 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.255 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254556 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:00:47.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816819+0000", 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.046e-06 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816825+0000", 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8860000000000001e-06 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816829+0000", 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873320+0000", 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056491373999999997 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873383+0000", 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2312999999999998e-05 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873503+0000", 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012061 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.259 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:47.261 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:47.262 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:47.253+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:47.262 INFO:teuthology.orchestra.run.smithi042.stdout:5496 trunc from 0x26a972b to 0x9cb0321 2024-09-16T13:00:47.262 INFO:teuthology.orchestra.run.smithi042.stdout:5497 write 0x84549a thru 0x84ec94 (0x97fb bytes) 2024-09-16T13:00:47.263 INFO:teuthology.orchestra.run.smithi042.stdout:5498 write 0x8a5b92d thru 0x8a5cc7b (0x134f bytes) 2024-09-16T13:00:47.263 INFO:teuthology.orchestra.run.smithi042.stdout:5500 read 0x9fda32 thru 0xa02022 (0x45f1 bytes) 2024-09-16T13:00:47.263 INFO:teuthology.orchestra.run.smithi042.stdout:5503 read 0x53c18a2 thru 0x53cb966 (0xa0c5 bytes) 2024-09-16T13:00:47.263 INFO:teuthology.orchestra.run.smithi042.stdout:5504 write 0xd24f33f thru 0xd25c399 (0xd05b bytes) 2024-09-16T13:00:47.263 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:47.263 INFO:teuthology.orchestra.run.smithi042.stdout:5506 read 0xb1d87da thru 0xb1e6c15 (0xe43c bytes) 2024-09-16T13:00:47.263 INFO:teuthology.orchestra.run.smithi042.stdout:5507 trunc from 0xd25c39a to 0x81b4447 2024-09-16T13:00:47.263 INFO:teuthology.orchestra.run.smithi042.stdout:5509 read 0x13b4ef3 thru 0x13bb2cd (0x63db bytes) 2024-09-16T13:00:47.263 INFO:teuthology.orchestra.run.smithi042.stdout:5510 read 0x6fb50e3 thru 0x6fbb356 (0x6274 bytes) 2024-09-16T13:00:47.263 INFO:teuthology.orchestra.run.smithi042.stdout:5511 trunc from 0x81b4447 to 0x76f8759 2024-09-16T13:00:47.266 INFO:teuthology.orchestra.run.smithi042.stdout:5512 write 0x1b30cf0 thru 0x1b3c114 (0xb425 bytes) 2024-09-16T13:00:47.290 INFO:teuthology.orchestra.run.smithi042.stdout:5513 write 0x976f6cd thru 0x977c1a8 (0xcadc bytes) 2024-09-16T13:00:47.297 INFO:teuthology.orchestra.run.smithi042.stdout:5515 read 0x330e475 thru 0x331c355 (0xdee1 bytes) 2024-09-16T13:00:47.297 INFO:teuthology.orchestra.run.smithi042.stdout:5517 punch from 0x83791ce to 0x8379d00, (0xb32 bytes) 2024-09-16T13:00:47.308 INFO:teuthology.orchestra.run.smithi042.stdout:5519 trunc from 0x977c1a9 to 0xe221c50 2024-09-16T13:00:47.318 INFO:teuthology.orchestra.run.smithi042.stdout:5522 punch from 0xc9fbf49 to 0xca0a654, (0xe70b bytes) 2024-09-16T13:00:47.318 INFO:teuthology.orchestra.run.smithi042.stdout:5523 read 0x5d7989c thru 0x5d7fcf5 (0x645a bytes) 2024-09-16T13:00:47.318 INFO:teuthology.orchestra.run.smithi042.stdout:5524 read 0x8836c4c thru 0x884403f (0xd3f4 bytes) 2024-09-16T13:00:47.318 INFO:teuthology.orchestra.run.smithi042.stdout:5525 punch from 0x4d27e57 to 0x4d362f7, (0xe4a0 bytes) 2024-09-16T13:00:47.318 INFO:teuthology.orchestra.run.smithi042.stdout:5526 read 0x6cf1a1 thru 0x6d658d (0x73ed bytes) 2024-09-16T13:00:47.330 INFO:teuthology.orchestra.run.smithi042.stdout:5527 write 0xeb55746 thru 0xeb5bc5d (0x6518 bytes) 2024-09-16T13:00:47.358 INFO:teuthology.orchestra.run.smithi042.stdout:5528 trunc from 0xeb5bc5e to 0x4b4a55f 2024-09-16T13:00:47.361 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:47.356+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:47.370 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:47.370 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:47.370 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:47.371 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:47.377 INFO:teuthology.orchestra.run.smithi042.stdout:5529 read 0x11940e5 thru 0x119e4e3 (0xa3ff bytes) 2024-09-16T13:00:47.380 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:47.406 INFO:teuthology.orchestra.run.smithi042.stdout:5532 write 0xcc1cf4d thru 0xcc2b69c (0xe750 bytes) 2024-09-16T13:00:47.433 INFO:teuthology.orchestra.run.smithi042.stdout:5533 trunc from 0xcc2b69d to 0x192f0c9 2024-09-16T13:00:47.450 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T13:00:47.462 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:47.456+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:47.479 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:47.479 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:47.488 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:47.563 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:47.557+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:47.586 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 122.03211629099999, 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:00:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:00:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.206999244, 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:00:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.206951988, 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:00:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:00:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:00:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:00:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:00:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.20674905200001, 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:00:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:00:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:00:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:00:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:00:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:00:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:00:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.20671779, 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:00:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:00:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:00:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.206682028, 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:00:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.20648739799999, 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:00:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.20644055299999, 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:00:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 120.662164382, 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:00:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 120.66211047500001, 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:00:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 120.66206817, 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:00:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:00:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 120.661909745, 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:00:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 120.661860264, 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:00:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 120.661729694, 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:47.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:00:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:00:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 120.661548181, 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.851020514, 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:00:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:00:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:00:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.85087156, 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:00:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64507 4.1e e83/31 4:7f7c7834:::rbd_data.11b5c2ee3542.0000000000000008:head v 83'225, mlcod=81'223)", 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.811458273, 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021284401000000001, 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770380+0000", 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8999999999999998e-07 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777141+0000", 2024-09-16T13:00:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067617670000000001 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777157+0000", 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777167+0000", 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777267+0000", 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100364 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777304+0000", 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7165000000000001e-05 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791601+0000", 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014297275999999999 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791664+0000", 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2217000000000004e-05 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64509 4.15 e83/31 4:ad2e18f4:::rbd_data.11b5c2ee3542.0000000000000009:head v 83'183, mlcod=81'180)", 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.804700370999999, 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014054395000000001, 2024-09-16T13:00:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777138+0000", 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778534+0000", 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013956769999999999 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778555+0000", 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1046000000000001e-05 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778564+0000", 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9290000000000004e-06 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778633+0000", 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9363000000000007e-05 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778666+0000", 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135000000000003e-05 2024-09-16T13:00:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791145+0000", 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012479743 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791192+0000", 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6156000000000003e-05 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:217754 4.17 e92/31 4:ee4048fa:::rbd_data.11b58362ce83.0000000000000002:head v 92'697, mlcod=91'693)", 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.082530317, 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011416822, 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499309+0000", 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6929999999999999e-06 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499322+0000", 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:00:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499354+0000", 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135999999999999e-05 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499365+0000", 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111e-05 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499504+0000", 2024-09-16T13:00:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013877599999999999 2024-09-16T13:00:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499565+0000", 2024-09-16T13:00:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1939000000000004e-05 2024-09-16T13:00:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510591+0000", 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011025874999999999 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510724+0000", 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013266299999999999 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:47.706 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:47.708 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:47.701+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:47.708 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:47.809 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:47.800+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:47.821 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:47.821 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:47.830 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:47.910 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:47.905+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:47.928 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:47.928 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:47.929 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:47.929 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:47.929 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:47.938 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:48.011 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:48.004+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349947258, 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.052 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349932232, 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:48.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:00:48.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349916210000004, 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349900114, 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:00:48.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349884224, 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.070 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349868553999997, 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349852806999998, 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:48.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349837018999999, 2024-09-16T13:00:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:00:48.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:00:48.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349821697000003, 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:00:48.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:00:48.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349801771999999, 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:00:48.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:00:48.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349785824000001, 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:00:48.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:00:48.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349770663000001, 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.097 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:48.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349755015, 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:00:48.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349739917999997, 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:48.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:00:48.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:48.110 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349724655999999, 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:00:48.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.114 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.115 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.115 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:48.115 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349708909999997, 2024-09-16T13:00:48.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:00:48.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:00:48.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.119 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349693313000003, 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:00:48.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349678009000002, 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:00:48.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349662571000003, 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:00:48.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.349646858, 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:00:48.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:00:48.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:00:48.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:00:48.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.138 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.138 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.138 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:48.140 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:48.140 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:48.132+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:48.240 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:48.233+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:48.290 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:48.291 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:48.300 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:48.341 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:48.336+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:48.404 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:48.404 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:48.413 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:48.442 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:48.437+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:48.469 INFO:teuthology.orchestra.run.smithi042.stdout:5534 read 0x13203b3 thru 0x1325fcd (0x5c1b bytes) 2024-09-16T13:00:48.478 INFO:teuthology.orchestra.run.smithi042.stdout:5535 trunc from 0x192f0c9 to 0x6c87857 2024-09-16T13:00:48.488 INFO:teuthology.orchestra.run.smithi042.stdout:5536 write 0x1e28e15 thru 0x1e29e6d (0x1059 bytes) 2024-09-16T13:00:48.510 INFO:teuthology.orchestra.run.smithi042.stdout:5537 write 0xcbc0f7a thru 0xcbc9f3d (0x8fc4 bytes) 2024-09-16T13:00:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:5538 punch from 0x8618078 to 0x8624956, (0xc8de bytes) 2024-09-16T13:00:48.537 INFO:teuthology.orchestra.run.smithi042.stdout:5540 trunc from 0xcbc9f3e to 0x3dce9e0 2024-09-16T13:00:48.543 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:48.537+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:48.559 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254136 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 136192~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7051166530000001, 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082717923999999998, 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254136 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849492+0000", 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-16T13:00:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849494+0000", 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0540000000000002e-06 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849500+0000", 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2899999999999999e-06 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849504+0000", 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8120000000000001e-06 2024-09-16T13:00:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.931987+0000", 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082483231000000004 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932061+0000", 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4181e-05 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932209+0000", 2024-09-16T13:00:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014796399999999999 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254137 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 139264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.705100378, 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082930443000000006, 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254137 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849510+0000", 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0530000000000001e-06 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849516+0000", 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8359999999999997e-06 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849520+0000", 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932239+0000", 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082719191999999997 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932307+0000", 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7824000000000007e-05 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932438+0000", 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000131354 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254138 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 146432~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.705084448, 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083100513000000001, 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254138 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7300000000000001e-07 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849526+0000", 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9089999999999998e-06 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849532+0000", 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6420000000000004e-06 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849535+0000", 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7740000000000002e-06 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932467+0000", 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082931469999999993 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932535+0000", 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7815e-05 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932624+0000", 2024-09-16T13:00:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9430000000000003e-05 2024-09-16T13:00:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254139 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 149504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:00:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7050688909999998, 2024-09-16T13:00:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083267068, 2024-09-16T13:00:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254139 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849540+0000", 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849542+0000", 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9379999999999999e-06 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849547+0000", 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:00:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849551+0000", 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5880000000000002e-06 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932652+0000", 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083101176999999998 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932719+0000", 2024-09-16T13:00:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.703e-05 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932806+0000", 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7340000000000001e-05 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254140 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 156672~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.705053618, 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083436222000000004, 2024-09-16T13:00:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254140 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:00:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849557+0000", 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8419999999999999e-06 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849564+0000", 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7079999999999997e-06 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849569+0000", 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.386e-06 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932833+0000", 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083264099999999994 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932900+0000", 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7007000000000004e-05 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932991+0000", 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0815000000000004e-05 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254141 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:00:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7050350659999998, 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083599532000000004, 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254141 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849574+0000", 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1699999999999998e-07 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849576+0000", 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849582+0000", 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9769999999999997e-06 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849585+0000", 2024-09-16T13:00:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933018+0000", 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083432372000000005 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933086+0000", 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7862999999999997e-05 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933173+0000", 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7050000000000002e-05 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254142 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7050189160000002, 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083767603999999996, 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254142 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:00:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849590+0000", 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.51e-07 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849592+0000", 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0360000000000001e-06 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849598+0000", 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4139999999999999e-06 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849602+0000", 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.659e-06 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933199+0000", 2024-09-16T13:00:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083597150999999995 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933267+0000", 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7933000000000002e-05 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933357+0000", 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9859999999999997e-05 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254143 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7050019680000004, 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083932426000000004, 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254143 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849607+0000", 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849609+0000", 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.165e-06 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849615+0000", 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2029999999999997e-06 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849619+0000", 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933382+0000", 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083763447000000005 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933447+0000", 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4993000000000006e-05 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933539+0000", 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1434000000000006e-05 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254145 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7049854060000005, 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084130242999999993, 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254145 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851596+0000", 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019732550000000001 2024-09-16T13:00:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851602+0000", 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5720000000000002e-06 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851610+0000", 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5529999999999993e-06 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851616+0000", 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6409999999999999e-06 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933580+0000", 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.081963854000000003 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933653+0000", 2024-09-16T13:00:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3901999999999998e-05 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933753+0000", 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9465999999999999e-05 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254149 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7029688830000005, 2024-09-16T13:00:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082661199000000005, 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254149 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851640+0000", 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851642+0000", 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851648+0000", 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9200000000000001e-06 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851652+0000", 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0439999999999997e-06 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934105+0000", 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082453495000000002 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934193+0000", 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7433000000000001e-05 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934300+0000", 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107513 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254151 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7029472620000004, 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082910496, 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254151 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:00:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1799999999999995e-07 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851664+0000", 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4399999999999999e-06 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851670+0000", 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3509999999999998e-06 2024-09-16T13:00:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851674+0000", 2024-09-16T13:00:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.912e-06 2024-09-16T13:00:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934385+0000", 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082710396000000005 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934471+0000", 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6040999999999998e-05 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934571+0000", 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100738 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254153 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7029298090000005, 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083065172000000007, 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254153 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851679+0000", 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851681+0000", 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.137e-06 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851687+0000", 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.643e-06 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851690+0000", 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5580000000000001e-06 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934597+0000", 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082906881000000002 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934663+0000", 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5828999999999996e-05 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934743+0000", 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0714e-05 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254155 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7029141210000001, 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083254350000000005, 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254155 2024-09-16T13:00:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851697+0000", 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.187e-06 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851702+0000", 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7520000000000002e-06 2024-09-16T13:00:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851706+0000", 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7979999999999999e-06 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934769+0000", 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083062834000000002 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934829+0000", 2024-09-16T13:00:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0430999999999998e-05 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934948+0000", 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118905 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254157 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 207872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7028981959999996, 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083456394000000003, 2024-09-16T13:00:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254157 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851712+0000", 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0150000000000002e-06 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851718+0000", 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5999999999999997e-06 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851722+0000", 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5810000000000001e-06 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934992+0000", 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083270799000000006 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935066+0000", 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3851999999999997e-05 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935166+0000", 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100145 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254159 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 210944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7028828499999999, 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083625606000000005, 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254159 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851726+0000", 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851728+0000", 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851734+0000", 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7429999999999999e-06 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851738+0000", 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9620000000000004e-06 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935195+0000", 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083457518999999994 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935262+0000", 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6465999999999997e-05 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935351+0000", 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9314000000000006e-05 2024-09-16T13:00:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254161 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 218112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7028668600000003, 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083866861000000001, 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254161 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:00:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851742+0000", 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851744+0000", 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1550000000000001e-06 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851749+0000", 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5829999999999997e-06 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851753+0000", 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6990000000000001e-06 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935377+0000", 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083624025000000005 2024-09-16T13:00:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935454+0000", 2024-09-16T13:00:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7168000000000004e-05 2024-09-16T13:00:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935608+0000", 2024-09-16T13:00:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153774 2024-09-16T13:00:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254163 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 221184~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7028510839999997, 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084055344000000004, 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254163 2024-09-16T13:00:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851760+0000", 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2249999999999999e-06 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851765+0000", 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7799999999999997e-06 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851769+0000", 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-06 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935654+0000", 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083884829999999994 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935723+0000", 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9163000000000002e-05 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935812+0000", 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9237e-05 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254164 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7028350410000002, 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084726810999999999, 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254164 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851775+0000", 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.945e-06 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851781+0000", 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851785+0000", 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6320000000000001e-06 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936053+0000", 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084268018 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936130+0000", 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7478999999999995e-05 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936500+0000", 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036965099999999997 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254165 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 228352~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.702819517, 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084960919999999995, 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254165 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:00:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851791+0000", 2024-09-16T13:00:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2819999999999999e-06 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851797+0000", 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7969999999999997e-06 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851801+0000", 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6679999999999999e-06 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936554+0000", 2024-09-16T13:00:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084753203999999999 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936625+0000", 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1406e-05 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936750+0000", 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012416700000000001 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254166 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 6144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7028037060000001, 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.085115273000000005, 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254166 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:00:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856445+0000", 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046400929999999996 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856451+0000", 2024-09-16T13:00:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5119999999999997e-06 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856463+0000", 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2252999999999999e-05 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856470+0000", 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4160000000000001e-06 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936767+0000", 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080297258999999996 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936832+0000", 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4685999999999997e-05 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936920+0000", 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8053999999999994e-05 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:48.654 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:48.654 INFO:teuthology.orchestra.run.smithi042.stdout:5541 write 0x863550d thru 0x863ab0c (0x5600 bytes) 2024-09-16T13:00:48.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:5544 writesame 0xd4c6034 thru 0xd4cc97e data_size 0xf(0x694b bytes) 2024-09-16T13:00:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:5545 trunc from 0xd4cc97f to 0xc5a8ec4 2024-09-16T13:00:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:5546 punch from 0x2f7a322 to 0x2f89a3f, (0xf71d bytes) 2024-09-16T13:00:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:5547 read 0xbc89a0e thru 0xbc904fa (0x6aed bytes) 2024-09-16T13:00:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:5548 trunc from 0xc5a8ec4 to 0xa286a8d 2024-09-16T13:00:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:5549 writesame 0x3938594 thru 0x3948419 data_size 0x1(0xfe86 bytes) 2024-09-16T13:00:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:5550 read 0x6193c9a thru 0x619f805 (0xbb6c bytes) 2024-09-16T13:00:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:5551 writesame 0x85c90a9 thru 0x85d36d9 data_size 0x5(0xa631 bytes) 2024-09-16T13:00:48.657 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:48.648+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:48.665 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:48.665 INFO:teuthology.orchestra.run.smithi042.stdout:5555 write 0x2d4258 thru 0x2e30ab (0xee54 bytes) 2024-09-16T13:00:48.694 INFO:teuthology.orchestra.run.smithi042.stdout:5556 trunc from 0xa286a8d to 0x3590fd 2024-09-16T13:00:48.749 INFO:teuthology.orchestra.run.smithi042.stdout:5557 trunc from 0x3590fd to 0xe0bb164 2024-09-16T13:00:48.756 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:48.748+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:48.758 INFO:teuthology.orchestra.run.smithi042.stdout:5559 write 0xc1729fd thru 0xc17a698 (0x7c9c bytes) 2024-09-16T13:00:48.764 INFO:teuthology.orchestra.run.smithi042.stdout:5560 read 0x4334b06 thru 0x433dc1f (0x911a bytes) 2024-09-16T13:00:48.764 INFO:teuthology.orchestra.run.smithi042.stdout:5561 read 0x533238f thru 0x533e3af (0xc021 bytes) 2024-09-16T13:00:48.764 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:48.764 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:48.764 INFO:teuthology.orchestra.run.smithi042.stdout:5564 read 0x4c531db thru 0x4c6023f (0xd065 bytes) 2024-09-16T13:00:48.764 INFO:teuthology.orchestra.run.smithi042.stdout:5566 write 0xa589a2d thru 0xa593244 (0x9818 bytes) 2024-09-16T13:00:48.770 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:48.770 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:48.782 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:48.783 INFO:teuthology.orchestra.run.smithi042.stdout:5567 trunc from 0xe0bb164 to 0x6fea8cf 2024-09-16T13:00:48.824 INFO:teuthology.orchestra.run.smithi042.stdout:5568 write 0xeb18a3c thru 0xeb21a58 (0x901d bytes) 2024-09-16T13:00:48.830 INFO:teuthology.orchestra.run.smithi042.stdout:5571 read 0xb2a0a6e thru 0xb2a301b (0x25ae bytes) 2024-09-16T13:00:48.830 INFO:teuthology.orchestra.run.smithi042.stdout:5572 writesame 0x890a424 thru 0x890c833 data_size 0x8(0x2410 bytes) 2024-09-16T13:00:48.840 INFO:teuthology.orchestra.run.smithi042.stdout:5573 punch from 0x3298d8a to 0x329a4fa, (0x1770 bytes) 2024-09-16T13:00:48.840 INFO:teuthology.orchestra.run.smithi042.stdout:5574 read 0xdf650d8 thru 0xdf6b268 (0x6191 bytes) 2024-09-16T13:00:48.840 INFO:teuthology.orchestra.run.smithi042.stdout:5575 writesame 0x1d1bc9c thru 0x1d1bf9a data_size 0x1(0x2ff bytes) 2024-09-16T13:00:48.843 INFO:teuthology.orchestra.run.smithi042.stdout:5576 write 0x2b16175 thru 0x2b19601 (0x348d bytes) 2024-09-16T13:00:48.850 INFO:teuthology.orchestra.run.smithi042.stdout:5578 writesame 0x4a427b5 thru 0x4a4825f data_size 0x3(0x5aab bytes) 2024-09-16T13:00:48.857 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:48.852+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:48.862 INFO:teuthology.orchestra.run.smithi042.stdout:5579 write 0x9a19196 thru 0x9a21633 (0x849e bytes) 2024-09-16T13:00:48.867 INFO:teuthology.orchestra.run.smithi042.stdout:5580 writesame 0x91a56e9 thru 0x91a961b data_size 0x1(0x3f33 bytes) 2024-09-16T13:00:48.876 INFO:teuthology.orchestra.run.smithi042.stdout:5582 read 0xab84850 thru 0xab90cb7 (0xc468 bytes) 2024-09-16T13:00:48.876 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:48.876 INFO:teuthology.orchestra.run.smithi042.stdout:5584 read 0x52d98bc thru 0x52e11ee (0x7933 bytes) 2024-09-16T13:00:48.876 INFO:teuthology.orchestra.run.smithi042.stdout:5585 write 0x2b0d18a thru 0x2b164f8 (0x936f bytes) 2024-09-16T13:00:48.880 INFO:teuthology.orchestra.run.smithi042.stdout:5586 punch from 0xcedb5a6 to 0xcedf4b5, (0x3f0f bytes) 2024-09-16T13:00:48.880 INFO:teuthology.orchestra.run.smithi042.stdout:5587 punch from 0x9b21e45 to 0x9b23ebd, (0x2078 bytes) 2024-09-16T13:00:48.880 INFO:teuthology.orchestra.run.smithi042.stdout:5588 read 0x6604275 thru 0x66131a9 (0xef35 bytes) 2024-09-16T13:00:48.880 INFO:teuthology.orchestra.run.smithi042.stdout:5589 write 0x9fcaffa thru 0x9fce3c6 (0x33cd bytes) 2024-09-16T13:00:48.884 INFO:teuthology.orchestra.run.smithi042.stdout:5590 punch from 0x3aad687 to 0x3abd230, (0xfba9 bytes) 2024-09-16T13:00:48.885 INFO:teuthology.orchestra.run.smithi042.stdout:5591 read 0x33add3e thru 0x33bb006 (0xd2c9 bytes) 2024-09-16T13:00:48.885 INFO:teuthology.orchestra.run.smithi042.stdout:5593 writesame 0xdb0a4ae thru 0xdb124ad data_size 0x20(0x8000 bytes) 2024-09-16T13:00:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:48.892 INFO:teuthology.orchestra.run.smithi042.stdout:5594 read 0xd47c557 thru 0xd47cd41 (0x7eb bytes) 2024-09-16T13:00:48.892 INFO:teuthology.orchestra.run.smithi042.stdout:5596 writesame 0x45c0c99 thru 0x45c0f26 data_size 0x3(0x28e bytes) 2024-09-16T13:00:48.897 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:48.897 INFO:teuthology.orchestra.run.smithi042.stdout:5597 write 0xe59af4c thru 0xe5a99d6 (0xea8b bytes) 2024-09-16T13:00:48.904 INFO:teuthology.orchestra.run.smithi042.stdout:5599 write 0x53dd928 thru 0x53ed493 (0xfb6c bytes) 2024-09-16T13:00:48.910 INFO:teuthology.orchestra.run.smithi042.stdout:5601 read 0x25e5ae1 thru 0x25f2836 (0xcd56 bytes) 2024-09-16T13:00:48.911 INFO:teuthology.orchestra.run.smithi042.stdout:5603 write 0x57c1f1c thru 0x57c24e1 (0x5c6 bytes) 2024-09-16T13:00:48.916 INFO:teuthology.orchestra.run.smithi042.stdout:5604 read 0xa45a261 thru 0xa45df44 (0x3ce4 bytes) 2024-09-16T13:00:48.916 INFO:teuthology.orchestra.run.smithi042.stdout:5605 writesame 0xe701c4 thru 0xe722cd data_size 0x1(0x210a bytes) 2024-09-16T13:00:48.926 INFO:teuthology.orchestra.run.smithi042.stdout:5606 write 0x65a77cb thru 0x65aa400 (0x2c36 bytes) 2024-09-16T13:00:48.933 INFO:teuthology.orchestra.run.smithi042.stdout:5608 writesame 0xc5680ee thru 0xc56e847 data_size 0x1(0x675a bytes) 2024-09-16T13:00:48.958 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:48.952+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:48.960 INFO:teuthology.orchestra.run.smithi042.stdout:5609 punch from 0x19f28dd to 0x19fd63f, (0xad62 bytes) 2024-09-16T13:00:48.963 INFO:teuthology.orchestra.run.smithi042.stdout:5610 trunc from 0xeb21a59 to 0xd1ba254 2024-09-16T13:00:48.973 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:48.983 INFO:teuthology.orchestra.run.smithi042.stdout:5611 writesame 0x726c3b9 thru 0x726c49c data_size 0x1(0xe4 bytes) 2024-09-16T13:00:48.988 INFO:teuthology.orchestra.run.smithi042.stdout:5612 punch from 0xc86e5a1 to 0xc87d1ef, (0xec4e bytes) 2024-09-16T13:00:48.997 INFO:teuthology.orchestra.run.smithi042.stdout:5613 writesame 0xd5c9383 thru 0xd5cfdc0 data_size 0x9(0x6a3e bytes) 2024-09-16T13:00:48.997 INFO:teuthology.orchestra.run.smithi042.stdout:5616 write 0x42b079e thru 0x42b3dd5 (0x3638 bytes) 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260177 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9232050709999999, 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039279541000000001, 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260177 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075845+0000", 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075847+0000", 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075852+0000", 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.507e-06 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075856+0000", 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6389999999999998e-06 2024-09-16T13:00:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.114965+0000", 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039109496000000001 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115034+0000", 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8378000000000002e-05 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115124+0000", 2024-09-16T13:00:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0334999999999995e-05 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260178 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9231896749999997, 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039530044, 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260178 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075866+0000", 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0909999999999998e-06 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075872+0000", 2024-09-16T13:00:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.139e-06 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075876+0000", 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0419999999999996e-06 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115194+0000", 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039317193 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115259+0000", 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5399000000000002e-05 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115390+0000", 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130803 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260179 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9231688250000003, 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039926426000000001, 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260179 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075883+0000", 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6959999999999999e-06 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075889+0000", 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9479999999999996e-06 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075893+0000", 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7840000000000001e-06 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115419+0000", 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039526517999999997 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115577+0000", 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015783999999999999 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115807+0000", 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023010399999999999 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260180 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9231530780000003, 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040245398000000002, 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260180 2024-09-16T13:00:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075897+0000", 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:00:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075899+0000", 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8169999999999999e-06 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075904+0000", 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075908+0000", 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5889999999999999e-06 2024-09-16T13:00:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115837+0000", 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039929193000000002 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115978+0000", 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014115800000000001 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116142+0000", 2024-09-16T13:00:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016352900000000001 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260181 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 180224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9231378709999998, 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040482388000000001, 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260181 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075914+0000", 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6759999999999999e-06 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075919+0000", 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2360000000000004e-06 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075922+0000", 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116171+0000", 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040248702999999997 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116239+0000", 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8391000000000003e-05 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116394+0000", 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000154575 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260182 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9231231600000003, 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042150168000000002, 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260182 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075927+0000", 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075929+0000", 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7859999999999999e-06 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075934+0000", 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7679999999999997e-06 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075938+0000", 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2590000000000001e-06 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116426+0000", 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040488077999999997 2024-09-16T13:00:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116493+0000", 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7681000000000005e-05 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118077+0000", 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015832260000000001 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260183 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9231081720000001, 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040886082999999997, 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260183 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075942+0000", 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075943+0000", 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.778e-06 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075949+0000", 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075953+0000", 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116672+0000", 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040719563 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116743+0000", 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0729000000000007e-05 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116827+0000", 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4364999999999996e-05 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260184 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9230930090000005, 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041043524999999997, 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260184 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075959+0000", 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6869999999999999e-06 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075964+0000", 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.242e-06 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075967+0000", 2024-09-16T13:00:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116850+0000", 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040882977000000001 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116916+0000", 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5413000000000005e-05 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117000+0000", 2024-09-16T13:00:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4431999999999994e-05 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260185 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9230777699999999, 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041177895999999999, 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260185 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075974+0000", 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0049999999999999e-06 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075980+0000", 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7470000000000002e-06 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075984+0000", 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117022+0000", 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041038789999999999 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117090+0000", 2024-09-16T13:00:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7712000000000005e-05 2024-09-16T13:00:49.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117150+0000", 2024-09-16T13:00:49.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9542e-05 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260257 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9230359159999999, 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041222406000000003, 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260257 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076017+0000", 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4990000000000001e-06 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076024+0000", 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076028+0000", 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9319999999999998e-06 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117162+0000", 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04113472 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117199+0000", 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6507999999999997e-05 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117236+0000", 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7023000000000001e-05 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.049 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260258 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9230173969999997, 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041343291999999997, 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260258 2024-09-16T13:00:49.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076035+0000", 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.058e-06 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076041+0000", 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.998e-06 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076044+0000", 2024-09-16T13:00:49.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3900000000000002e-06 2024-09-16T13:00:49.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117256+0000", 2024-09-16T13:00:49.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041212555999999997 2024-09-16T13:00:49.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117323+0000", 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6568000000000004e-05 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117375+0000", 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2361999999999998e-05 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260259 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9230019790000004, 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041437587999999997, 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260259 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5499999999999999e-07 2024-09-16T13:00:49.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076050+0000", 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076055+0000", 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5609999999999998e-06 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076059+0000", 2024-09-16T13:00:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117396+0000", 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041336879999999999 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117454+0000", 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8443999999999998e-05 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117485+0000", 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1047999999999998e-05 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260260 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9229871799999998, 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041496984000000001, 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260260 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:00:49.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076063+0000", 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076070+0000", 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7919999999999993e-06 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076076+0000", 2024-09-16T13:00:49.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9359999999999996e-06 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076080+0000", 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8240000000000001e-06 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117497+0000", 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041416896000000002 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117531+0000", 2024-09-16T13:00:49.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4e-05 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117559+0000", 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8177000000000001e-05 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260261 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9229649440000003, 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041555793000000001, 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260261 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076087+0000", 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7850000000000001e-06 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076093+0000", 2024-09-16T13:00:49.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6879999999999996e-06 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076096+0000", 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117570+0000", 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041474329999999997 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117611+0000", 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0852999999999997e-05 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117640+0000", 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9213e-05 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260262 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:49.067 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9229494980000004, 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041643056999999997, 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260262 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076102+0000", 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8509999999999999e-06 2024-09-16T13:00:49.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076108+0000", 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8560000000000003e-06 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076111+0000", 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0620000000000001e-06 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117651+0000", 2024-09-16T13:00:49.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041539980999999997 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117703+0000", 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1975000000000003e-05 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117743+0000", 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9872e-05 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260263 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9229345169999998, 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041724412000000002, 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260263 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076117+0000", 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.762e-06 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076123+0000", 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076126+0000", 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.427e-06 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117757+0000", 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041631230999999998 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117792+0000", 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4078999999999997e-05 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117839+0000", 2024-09-16T13:00:49.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7861000000000001e-05 2024-09-16T13:00:49.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.076 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.076 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260264 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9229196709999998, 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041783638999999997, 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260264 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:49.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076132+0000", 2024-09-16T13:00:49.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.751e-06 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076138+0000", 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6729999999999997e-06 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076141+0000", 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117851+0000", 2024-09-16T13:00:49.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04171064 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117885+0000", 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3807999999999999e-05 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117913+0000", 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8229999999999999e-05 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260265 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9229050680000004, 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042095026000000001, 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260265 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076145+0000", 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076147+0000", 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076152+0000", 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5130000000000004e-06 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076156+0000", 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117924+0000", 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041768179000000002 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117980+0000", 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6125000000000002e-05 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118239+0000", 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025934200000000003 2024-09-16T13:00:49.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260266 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9228902000000003, 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045273031999999998, 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260266 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076160+0000", 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076161+0000", 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.796e-06 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076167+0000", 2024-09-16T13:00:49.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-06 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076171+0000", 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.579e-06 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.120966+0000", 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.044795297999999997 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121159+0000", 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192618 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121432+0000", 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000273663 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.089 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260267 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.9228738229999998, 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046181535000000003, 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260267 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:49.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076178+0000", 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7969999999999999e-06 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076184+0000", 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6570000000000002e-06 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076187+0000", 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.455e-06 2024-09-16T13:00:49.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122253+0000", 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046065813999999997 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122318+0000", 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5338000000000004e-05 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122357+0000", 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9088999999999998e-05 2024-09-16T13:00:49.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.094 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:49.094 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:49.094 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:49.094 INFO:teuthology.orchestra.run.smithi042.stdout:5617 read 0x2305ff0 thru 0x2309dea (0x3dfb bytes) 2024-09-16T13:00:49.094 INFO:teuthology.orchestra.run.smithi042.stdout:5619 write 0xd96e539 thru 0xd97c6e5 (0xe1ad bytes) 2024-09-16T13:00:49.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:49.097 INFO:teuthology.orchestra.run.smithi042.stdout:5620 read 0x5848e10 thru 0x584e49f (0x5690 bytes) 2024-09-16T13:00:49.097 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:49.097 INFO:teuthology.orchestra.run.smithi042.stdout:5622 punch from 0x8272a27 to 0x82810a3, (0xe67c bytes) 2024-09-16T13:00:49.097 INFO:teuthology.orchestra.run.smithi042.stdout:5624 write 0x37f004a thru 0x37f3f33 (0x3eea bytes) 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:5626 write 0xd2baf69 thru 0xd2c349c (0x8534 bytes) 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:5627 read 0xb15d32a thru 0xb15dc6c (0x943 bytes) 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:5629 writesame 0xba2612e thru 0xba28910 data_size 0x1(0x27e3 bytes) 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:5630 trunc from 0xd97c6e6 to 0x73e614d 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:5632 punch from 0x2e1457d to 0x2e22294, (0xdd17 bytes) 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:5633 read 0x6d6858e thru 0x6d7854b (0xffbe bytes) 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:5634 write 0x8764f84 thru 0x87679e9 (0x2a66 bytes) 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:5638 read 0x5af121e thru 0x5af9344 (0x8127 bytes) 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:5641 punch from 0x550b66f to 0x5519fb1, (0xe942 bytes) 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:49.098 INFO:teuthology.orchestra.run.smithi042.stdout:5643 trunc from 0x87679ea to 0xb553425 2024-09-16T13:00:49.099 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:49.089+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:49.099 INFO:teuthology.orchestra.run.smithi042.stdout:5645 read 0x9623344 thru 0x9628c01 (0x58be bytes) 2024-09-16T13:00:49.100 INFO:teuthology.orchestra.run.smithi042.stdout:5647 punch from 0xa4dde16 to 0xa4e0c56, (0x2e40 bytes) 2024-09-16T13:00:49.100 INFO:teuthology.orchestra.run.smithi042.stdout:5649 write 0x46bcf14 thru 0x46c0c02 (0x3cef bytes) 2024-09-16T13:00:49.104 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:49.105 INFO:teuthology.orchestra.run.smithi042.stdout:5650 read 0x36c86ce thru 0x36c9c2a (0x155d bytes) 2024-09-16T13:00:49.106 INFO:teuthology.orchestra.run.smithi042.stdout:5651 read 0x59f83dc thru 0x59fb4c6 (0x30eb bytes) 2024-09-16T13:00:49.106 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:49.106 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:49.106 INFO:teuthology.orchestra.run.smithi042.stdout:5656 clone 36 order 18 su 8192 sc 8 2024-09-16T13:00:49.123 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:49.144 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:49.164 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:49.188 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:49.196 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:49.196 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:49.198 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:49.193+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:49.206 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:49.211 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:49.234 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:49.234 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:49.234 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:49.234 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:49.234 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:49.235 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:49.235 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:49.235 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:49.235 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:49.238 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:49.238 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:49.300 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:49.293+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:49.310 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:49.310 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:49.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:49.401 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:49.396+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:49.419 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:49.419 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:49.429 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:49.463 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-16T13:00:49.463 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-16T13:00:49.463 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T13:00:49.463 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T13:00:49.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T13:00:49.475 INFO:tasks.ceph.osd.4:Started 2024-09-16T13:00:49.475 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:00:49.502 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:49.493+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:49.543 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:49.544 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:49.555 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:49.571 INFO:teuthology.orchestra.run.smithi136.stderr:no valid command found; 10 closest matches: 2024-09-16T13:00:49.571 INFO:teuthology.orchestra.run.smithi136.stderr:0 2024-09-16T13:00:49.571 INFO:teuthology.orchestra.run.smithi136.stderr:1 2024-09-16T13:00:49.571 INFO:teuthology.orchestra.run.smithi136.stderr:2 2024-09-16T13:00:49.571 INFO:teuthology.orchestra.run.smithi136.stderr:abort 2024-09-16T13:00:49.572 INFO:teuthology.orchestra.run.smithi136.stderr:assert 2024-09-16T13:00:49.572 INFO:teuthology.orchestra.run.smithi136.stderr:config diff 2024-09-16T13:00:49.572 INFO:teuthology.orchestra.run.smithi136.stderr:config diff get 2024-09-16T13:00:49.572 INFO:teuthology.orchestra.run.smithi136.stderr:config get 2024-09-16T13:00:49.572 INFO:teuthology.orchestra.run.smithi136.stderr:config help [] 2024-09-16T13:00:49.572 INFO:teuthology.orchestra.run.smithi136.stderr:config set ... 2024-09-16T13:00:49.572 INFO:teuthology.orchestra.run.smithi136.stderr:admin_socket: invalid command 2024-09-16T13:00:49.573 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T13:00:49.573 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-16T13:00:49.602 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:49.596+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:49.642 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:49.642 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:49.651 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:49.703 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:49.696+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:49.746 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:49.746 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:49.755 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:49.803 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone35 from 0xb553425 (overlap 0x3590fd) to 0x156d32 2024-09-16T13:00:49.804 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:49.796+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:49.848 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:49.841+0000 7f466bedb540 -1 Falling back to public interface 2024-09-16T13:00:49.862 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:49.862 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:49.870 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:49.914 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:49.909+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:49.940 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-16T13:00:49.943 INFO:teuthology.orchestra.run.smithi042.stdout:5657 write 0x69992e9 thru 0x69a641b (0xd133 bytes) 2024-09-16T13:00:49.967 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:49.967 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:49.973 INFO:teuthology.orchestra.run.smithi042.stdout:5659 write 0x7bb1638 thru 0x7bb3069 (0x1a32 bytes) 2024-09-16T13:00:49.977 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:49.980 INFO:teuthology.orchestra.run.smithi042.stdout:5660 read 0x1d20379 thru 0x1d27603 (0x728b bytes) 2024-09-16T13:00:49.981 INFO:teuthology.orchestra.run.smithi042.stdout:5661 read 0x45cb871 thru 0x45ceaab (0x323b bytes) 2024-09-16T13:00:49.982 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:49.982 INFO:teuthology.orchestra.run.smithi042.stdout:5664 punch from 0x2c5e3ed to 0x2c633e5, (0x4ff8 bytes) 2024-09-16T13:00:49.982 INFO:teuthology.orchestra.run.smithi042.stdout:5665 read 0x1dd56c1 thru 0x1de38da (0xe21a bytes) 2024-09-16T13:00:49.982 INFO:teuthology.orchestra.run.smithi042.stdout:5667 read 0x1a489ba thru 0x1a5252a (0x9b71 bytes) 2024-09-16T13:00:49.982 INFO:teuthology.orchestra.run.smithi042.stdout:5668 clone 37 order 21 su 16384 sc 9 2024-09-16T13:00:50.006 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:49.997+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:50.067 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:50.068 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:50.077 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:50.107 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:50.100+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:50.208 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:50.200+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:50.221 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:50.221 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:50.230 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:50.310 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:50.304+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:50.327 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:50.327 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:50.336 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:50.410 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:50.408+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:50.431 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:50.431 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:50.440 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:50.483 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T13:00:50.541 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:50.536+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:50.560 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:50.560 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:50.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:50.612 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:50.604+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:50.627 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone36 intact 2024-09-16T13:00:50.671 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:50.671 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:50.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:50.713 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:50.709+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:50.784 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:50.784 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:50.794 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:50.814 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:50.809+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:50.859 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-16T13:00:50.872 INFO:teuthology.orchestra.run.smithi042.stdout:5669 write 0x563142e thru 0x563ec52 (0xd825 bytes) 2024-09-16T13:00:50.904 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:50.904 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:50.913 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:50.915 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:50.908+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:50.919 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:50.920 INFO:teuthology.orchestra.run.smithi042.stdout:5672 read 0x105267d thru 0x10562e2 (0x3c66 bytes) 2024-09-16T13:00:50.920 INFO:teuthology.orchestra.run.smithi042.stdout:5673 write 0xa9cdb21 thru 0xa9d1ceb (0x41cb bytes) 2024-09-16T13:00:50.930 INFO:teuthology.orchestra.run.smithi042.stdout:5674 read 0x1d2e6a1 thru 0x1d34706 (0x6066 bytes) 2024-09-16T13:00:50.943 INFO:teuthology.orchestra.run.smithi042.stdout:5675 write 0xcb5fa04 thru 0xcb6c046 (0xc643 bytes) 2024-09-16T13:00:50.959 INFO:teuthology.orchestra.run.smithi042.stdout:5676 punch from 0x8a843a1 to 0x8a919b4, (0xd613 bytes) 2024-09-16T13:00:50.959 INFO:teuthology.orchestra.run.smithi042.stdout:5678 write 0xc1faf70 thru 0xc2063db (0xb46c bytes) 2024-09-16T13:00:50.975 INFO:teuthology.orchestra.run.smithi042.stdout:5679 punch from 0x2dcfae3 to 0x2dd09a7, (0xec4 bytes) 2024-09-16T13:00:50.975 INFO:teuthology.orchestra.run.smithi042.stdout:5681 writesame 0xa59d9c6 thru 0xa5a9af5 data_size 0x58(0xc130 bytes) 2024-09-16T13:00:50.994 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:50.994 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:50.994 INFO:teuthology.orchestra.run.smithi042.stdout:5685 read 0xd8b7c6 thru 0xd9833c (0xcb77 bytes) 2024-09-16T13:00:51.009 INFO:teuthology.orchestra.run.smithi042.stdout:5686 trunc from 0xcb6c047 to 0x34b0422 2024-09-16T13:00:51.016 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:51.008+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:51.023 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:51.023 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:51.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:51.074 INFO:teuthology.orchestra.run.smithi042.stdout:5687 punch from 0x1156bcb to 0x11669fa, (0xfe2f bytes) 2024-09-16T13:00:51.074 INFO:teuthology.orchestra.run.smithi042.stdout:5688 read 0xb54d1d thru 0xb6447f (0xf763 bytes) 2024-09-16T13:00:51.074 INFO:teuthology.orchestra.run.smithi042.stdout:5689 read 0x2d2f76e thru 0x2d30ff4 (0x1887 bytes) 2024-09-16T13:00:51.075 INFO:teuthology.orchestra.run.smithi042.stdout:5690 punch from 0x10eb3be to 0x10f5f98, (0xabda bytes) 2024-09-16T13:00:51.075 INFO:teuthology.orchestra.run.smithi042.stdout:5691 read 0x31dc23b thru 0x31eb519 (0xf2df bytes) 2024-09-16T13:00:51.076 INFO:teuthology.orchestra.run.smithi042.stdout:5692 punch from 0x2572c4c to 0x2575516, (0x28ca bytes) 2024-09-16T13:00:51.076 INFO:teuthology.orchestra.run.smithi042.stdout:5693 read 0x241ecad thru 0x242d151 (0xe4a5 bytes) 2024-09-16T13:00:51.077 INFO:teuthology.orchestra.run.smithi042.stdout:5694 read 0x2049d3c thru 0x20518bb (0x7b80 bytes) 2024-09-16T13:00:51.087 INFO:teuthology.orchestra.run.smithi042.stdout:5696 trunc from 0x34b0422 to 0x8f290f0 2024-09-16T13:00:51.099 INFO:teuthology.orchestra.run.smithi042.stdout:5698 trunc from 0x8f290f0 to 0x796d89c 2024-09-16T13:00:51.111 INFO:teuthology.orchestra.run.smithi042.stdout:5699 read 0x3f2d975 thru 0x3f3a5a3 (0xcc2f bytes) 2024-09-16T13:00:51.111 INFO:teuthology.orchestra.run.smithi042.stdout:5700 write 0x4b61f42 thru 0x4b662e3 (0x43a2 bytes) 2024-09-16T13:00:51.116 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:51.109+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:51.120 INFO:teuthology.orchestra.run.smithi042.stdout:5701 read 0x5ec0d53 thru 0x5ecd3b6 (0xc664 bytes) 2024-09-16T13:00:51.120 INFO:teuthology.orchestra.run.smithi042.stdout:5702 read 0x65df73e thru 0x65e625a (0x6b1d bytes) 2024-09-16T13:00:51.128 INFO:teuthology.orchestra.run.smithi042.stdout:5704 trunc from 0x796d89c to 0xbe23bb6 2024-09-16T13:00:51.139 INFO:teuthology.orchestra.run.smithi042.stdout:5705 read 0xb0ca749 thru 0xb0d5d63 (0xb61b bytes) 2024-09-16T13:00:51.141 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:51.141 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:51.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:51.157 INFO:teuthology.orchestra.run.smithi042.stdout:5706 trunc from 0xbe23bb6 to 0x5d6737 2024-09-16T13:00:51.217 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:51.209+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:51.229 INFO:teuthology.orchestra.run.smithi042.stdout:5707 punch from 0x39e63 to 0x3e38a, (0x4527 bytes) 2024-09-16T13:00:51.240 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:51.248 INFO:teuthology.orchestra.run.smithi042.stdout:5708 writesame 0x718a7ce thru 0x7193e91 data_size 0x2(0x96c4 bytes) 2024-09-16T13:00:51.248 INFO:teuthology.orchestra.run.smithi042.stdout:5710 write 0x44a4ab9 thru 0x44ae7e7 (0x9d2f bytes) 2024-09-16T13:00:51.258 INFO:teuthology.orchestra.run.smithi042.stdout:5711 write 0x167d933 thru 0x168cf25 (0xf5f3 bytes) 2024-09-16T13:00:51.273 INFO:teuthology.orchestra.run.smithi042.stdout:5712 read 0x2d178cb thru 0x2d1d7c1 (0x5ef7 bytes) 2024-09-16T13:00:51.277 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:51.277 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:51.287 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:51.288 INFO:teuthology.orchestra.run.smithi042.stdout:5713 writesame 0x95d12b5 thru 0x95e1094 data_size 0x30(0xfde0 bytes) 2024-09-16T13:00:51.294 INFO:teuthology.orchestra.run.smithi042.stdout:5714 trunc from 0x95e1095 to 0x61e6d6e 2024-09-16T13:00:51.318 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:51.312+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:51.327 INFO:teuthology.orchestra.run.smithi042.stdout:5715 writesame 0xecc0ea9 thru 0xecc81c4 data_size 0x35(0x731c bytes) 2024-09-16T13:00:51.328 INFO:teuthology.orchestra.run.smithi042.stdout:5716 read 0x1d4f6d6 thru 0x1d54d55 (0x5680 bytes) 2024-09-16T13:00:51.328 INFO:teuthology.orchestra.run.smithi042.stdout:5717 read 0x8df1af9 thru 0x8df589a (0x3da2 bytes) 2024-09-16T13:00:51.328 INFO:teuthology.orchestra.run.smithi042.stdout:5718 write 0xab4aea8 thru 0xab52214 (0x736d bytes) 2024-09-16T13:00:51.342 INFO:teuthology.orchestra.run.smithi042.stdout:5719 read 0xb676ee3 thru 0xb680024 (0x9142 bytes) 2024-09-16T13:00:51.343 INFO:teuthology.orchestra.run.smithi042.stdout:5720 write 0x2b4b57d thru 0x2b4eb10 (0x3594 bytes) 2024-09-16T13:00:51.350 INFO:teuthology.orchestra.run.smithi042.stdout:5722 write 0x10c823f thru 0x10d5246 (0xd008 bytes) 2024-09-16T13:00:51.368 INFO:teuthology.orchestra.run.smithi042.stdout:5723 trunc from 0xecc81c5 to 0x67d54cf 2024-09-16T13:00:51.388 INFO:teuthology.orchestra.run.smithi042.stdout:5724 writesame 0x5b1f2b7 thru 0x5b2e54d data_size 0x7f(0xf297 bytes) 2024-09-16T13:00:51.391 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:51.399 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:51.399 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:51.406 INFO:teuthology.orchestra.run.smithi042.stdout:5725 read 0x4d35cb9 thru 0x4d3798c (0x1cd4 bytes) 2024-09-16T13:00:51.406 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:51.406 INFO:teuthology.orchestra.run.smithi042.stdout:5727 write 0x5c67ca2 thru 0x5c74155 (0xc4b4 bytes) 2024-09-16T13:00:51.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:51.411 INFO:teuthology.orchestra.run.smithi042.stdout:5728 read 0x1d94c90 thru 0x1da1c24 (0xcf95 bytes) 2024-09-16T13:00:51.413 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:51.418 INFO:teuthology.orchestra.run.smithi042.stdout:5731 trunc from 0x67d54cf to 0x2891717 2024-09-16T13:00:51.419 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:51.412+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:51.437 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:51.446 INFO:teuthology.orchestra.run.smithi042.stdout:5733 trunc from 0x2891717 to 0x6ff8151 2024-09-16T13:00:51.457 INFO:teuthology.orchestra.run.smithi042.stdout:5734 trunc from 0x6ff8151 to 0x6e8223b 2024-09-16T13:00:51.463 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:51.466 INFO:teuthology.orchestra.run.smithi042.stdout:5735 read 0x1c55c40 thru 0x1c5820c (0x25cd bytes) 2024-09-16T13:00:51.485 INFO:teuthology.orchestra.run.smithi042.stdout:5736 write 0xa2a15d4 thru 0xa2b04e3 (0xef10 bytes) 2024-09-16T13:00:51.493 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:51.503 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:51.514 INFO:teuthology.orchestra.run.smithi042.stdout:5738 writesame 0xb4ef07f thru 0xb4f48b5 data_size 0x1(0x5837 bytes) 2024-09-16T13:00:51.514 INFO:teuthology.orchestra.run.smithi042.stdout:5739 read 0x8c1c990 thru 0x8c2b210 (0xe881 bytes) 2024-09-16T13:00:51.517 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:51.519 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:51.519 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:51.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:51.519 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:51.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260177 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.438438734, 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039279541000000001, 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260177 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075845+0000", 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075847+0000", 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075852+0000", 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.507e-06 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075856+0000", 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6389999999999998e-06 2024-09-16T13:00:51.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.114965+0000", 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039109496000000001 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115034+0000", 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8378000000000002e-05 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115124+0000", 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0334999999999995e-05 2024-09-16T13:00:51.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260178 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4384233379999998, 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039530044, 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.524 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260178 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075866+0000", 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0909999999999998e-06 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075872+0000", 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.139e-06 2024-09-16T13:00:51.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075876+0000", 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0419999999999996e-06 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115194+0000", 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039317193 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115259+0000", 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5399000000000002e-05 2024-09-16T13:00:51.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115390+0000", 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130803 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260179 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4384024880000004, 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039926426000000001, 2024-09-16T13:00:51.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260179 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075883+0000", 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6959999999999999e-06 2024-09-16T13:00:51.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075889+0000", 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9479999999999996e-06 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075893+0000", 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7840000000000001e-06 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115419+0000", 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039526517999999997 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115577+0000", 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015783999999999999 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115807+0000", 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023010399999999999 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260180 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.532 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4383867410000004, 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040245398000000002, 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260180 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:00:51.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075897+0000", 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075899+0000", 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8169999999999999e-06 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075904+0000", 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075908+0000", 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5889999999999999e-06 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115837+0000", 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039929193000000002 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115978+0000", 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014115800000000001 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116142+0000", 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016352900000000001 2024-09-16T13:00:51.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260181 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 180224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4383715339999998, 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040482388000000001, 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260181 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075914+0000", 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6759999999999999e-06 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075919+0000", 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2360000000000004e-06 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075922+0000", 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116171+0000", 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040248702999999997 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116239+0000", 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8391000000000003e-05 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116394+0000", 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000154575 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260182 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4383568230000003, 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042150168000000002, 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260182 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075927+0000", 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075929+0000", 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7859999999999999e-06 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075934+0000", 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7679999999999997e-06 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075938+0000", 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2590000000000001e-06 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116426+0000", 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040488077999999997 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116493+0000", 2024-09-16T13:00:51.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7681000000000005e-05 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118077+0000", 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015832260000000001 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260183 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4383418350000001, 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040886082999999997, 2024-09-16T13:00:51.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260183 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075942+0000", 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075943+0000", 2024-09-16T13:00:51.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.778e-06 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075949+0000", 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075953+0000", 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116672+0000", 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040719563 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116743+0000", 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0729000000000007e-05 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116827+0000", 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4364999999999996e-05 2024-09-16T13:00:51.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260184 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4383266719999996, 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041043524999999997, 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260184 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075959+0000", 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6869999999999999e-06 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075964+0000", 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.242e-06 2024-09-16T13:00:51.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075967+0000", 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116850+0000", 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040882977000000001 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116916+0000", 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5413000000000005e-05 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117000+0000", 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4431999999999994e-05 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260185 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:51.554 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.438311433, 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041177895999999999, 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260185 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075974+0000", 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0049999999999999e-06 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075980+0000", 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7470000000000002e-06 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075984+0000", 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117022+0000", 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041038789999999999 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117090+0000", 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7712000000000005e-05 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117150+0000", 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9542e-05 2024-09-16T13:00:51.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260257 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.438269579, 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041222406000000003, 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260257 2024-09-16T13:00:51.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076017+0000", 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4990000000000001e-06 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076024+0000", 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076028+0000", 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9319999999999998e-06 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117162+0000", 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04113472 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117199+0000", 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6507999999999997e-05 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117236+0000", 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7023000000000001e-05 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260258 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4382510599999998, 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041343291999999997, 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260258 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076035+0000", 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.058e-06 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076041+0000", 2024-09-16T13:00:51.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.998e-06 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076044+0000", 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3900000000000002e-06 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117256+0000", 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041212555999999997 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117323+0000", 2024-09-16T13:00:51.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6568000000000004e-05 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117375+0000", 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2361999999999998e-05 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260259 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4382356420000004, 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041437587999999997, 2024-09-16T13:00:51.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260259 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5499999999999999e-07 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076050+0000", 2024-09-16T13:00:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076055+0000", 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5609999999999998e-06 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076059+0000", 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117396+0000", 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041336879999999999 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117454+0000", 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8443999999999998e-05 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117485+0000", 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1047999999999998e-05 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260260 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4382208429999999, 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041496984000000001, 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260260 2024-09-16T13:00:51.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:00:51.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076063+0000", 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076070+0000", 2024-09-16T13:00:51.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7919999999999993e-06 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076076+0000", 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9359999999999996e-06 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076080+0000", 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8240000000000001e-06 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117497+0000", 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041416896000000002 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117531+0000", 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4e-05 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117559+0000", 2024-09-16T13:00:51.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8177000000000001e-05 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260261 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4381986070000004, 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041555793000000001, 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260261 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076087+0000", 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7850000000000001e-06 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076093+0000", 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6879999999999996e-06 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076096+0000", 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117570+0000", 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041474329999999997 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117611+0000", 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0852999999999997e-05 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117640+0000", 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9213e-05 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260262 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4381831609999995, 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041643056999999997, 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260262 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076102+0000", 2024-09-16T13:00:51.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8509999999999999e-06 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076108+0000", 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8560000000000003e-06 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076111+0000", 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0620000000000001e-06 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117651+0000", 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041539980999999997 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117703+0000", 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1975000000000003e-05 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117743+0000", 2024-09-16T13:00:51.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9872e-05 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260263 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4381681799999999, 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041724412000000002, 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.587 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260263 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076117+0000", 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.762e-06 2024-09-16T13:00:51.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076123+0000", 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076126+0000", 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.427e-06 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117757+0000", 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041631230999999998 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117792+0000", 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4078999999999997e-05 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117839+0000", 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7861000000000001e-05 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260264 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4381533339999999, 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041783638999999997, 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260264 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076132+0000", 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.751e-06 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076138+0000", 2024-09-16T13:00:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6729999999999997e-06 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076141+0000", 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117851+0000", 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04171064 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117885+0000", 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3807999999999999e-05 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117913+0000", 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8229999999999999e-05 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260265 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4381387310000004, 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042095026000000001, 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260265 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.597 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076145+0000", 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076147+0000", 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076152+0000", 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5130000000000004e-06 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076156+0000", 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117924+0000", 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041768179000000002 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117980+0000", 2024-09-16T13:00:51.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6125000000000002e-05 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118239+0000", 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025934200000000003 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260266 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:00:51.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4381238630000004, 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045273031999999998, 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260266 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076160+0000", 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076161+0000", 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.796e-06 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076167+0000", 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-06 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076171+0000", 2024-09-16T13:00:51.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.579e-06 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.120966+0000", 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.044795297999999997 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121159+0000", 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192618 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121432+0000", 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000273663 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260267 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.4381074859999998, 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046181535000000003, 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:51.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260267 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076178+0000", 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7969999999999999e-06 2024-09-16T13:00:51.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076184+0000", 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6570000000000002e-06 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076187+0000", 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.455e-06 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122253+0000", 2024-09-16T13:00:51.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046065813999999997 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122318+0000", 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5338000000000004e-05 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122357+0000", 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9088999999999998e-05 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:51.611 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:51.611 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:51.611 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:51.512+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:51.611 INFO:teuthology.orchestra.run.smithi042.stdout:5740 write 0xd70187e thru 0xd70246f (0xbf2 bytes) 2024-09-16T13:00:51.612 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:51.613 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:51.613 INFO:teuthology.orchestra.run.smithi042.stdout:5743 trunc from 0xd702470 to 0xc36799d 2024-09-16T13:00:51.613 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:51.614 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:51.614 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:51.614 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:51.614 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:51.614 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:51.614 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:51.614 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:51.614 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:51.615 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:51.615 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:51.616 INFO:teuthology.orchestra.run.smithi042.stdout:5744 write 0xca28534 thru 0xca322fa (0x9dc7 bytes) 2024-09-16T13:00:51.616 INFO:teuthology.orchestra.run.smithi042.stdout:5745 read 0xc1420b4 thru 0xc1510fb (0xf048 bytes) 2024-09-16T13:00:51.616 INFO:teuthology.orchestra.run.smithi042.stdout:5746 read 0x77f7e6c thru 0x78074a8 (0xf63d bytes) 2024-09-16T13:00:51.616 INFO:teuthology.orchestra.run.smithi042.stdout:5747 read 0x8780a98 thru 0x878e204 (0xd76d bytes) 2024-09-16T13:00:51.617 INFO:teuthology.orchestra.run.smithi042.stdout:5748 writesame 0x9c73902 thru 0x9c7c596 data_size 0x1d(0x8c95 bytes) 2024-09-16T13:00:51.617 INFO:teuthology.orchestra.run.smithi042.stdout:5749 read 0x24214b1 thru 0x242346b (0x1fbb bytes) 2024-09-16T13:00:51.617 INFO:teuthology.orchestra.run.smithi042.stdout:5750 write 0x7a72442 thru 0x7a7541c (0x2fdb bytes) 2024-09-16T13:00:51.617 INFO:teuthology.orchestra.run.smithi042.stdout:5751 write 0xbc2a615 thru 0xbc2df1e (0x390a bytes) 2024-09-16T13:00:51.617 INFO:teuthology.orchestra.run.smithi042.stdout:5752 read 0x5814103 thru 0x5814a4e (0x94c bytes) 2024-09-16T13:00:51.617 INFO:teuthology.orchestra.run.smithi042.stdout:5754 write 0x3e9bca3 thru 0x3ea81d0 (0xc52e bytes) 2024-09-16T13:00:51.630 INFO:teuthology.orchestra.run.smithi042.stdout:5756 read 0x1b9ad73 thru 0x1b9b711 (0x99f bytes) 2024-09-16T13:00:51.630 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:51.624+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:51.631 INFO:teuthology.orchestra.run.smithi042.stdout:5757 write 0x729ca0f thru 0x72a90bd (0xc6af bytes) 2024-09-16T13:00:51.641 INFO:teuthology.orchestra.run.smithi042.stdout:5760 read 0xa66b492 thru 0xa66fef7 (0x4a66 bytes) 2024-09-16T13:00:51.642 INFO:teuthology.orchestra.run.smithi042.stdout:5762 writesame 0x2ac520f thru 0x2acf335 data_size 0x25(0xa127 bytes) 2024-09-16T13:00:51.667 INFO:teuthology.orchestra.run.smithi042.stdout:5764 trunc from 0xca322fb to 0x538b40e 2024-09-16T13:00:51.691 INFO:teuthology.orchestra.run.smithi042.stdout:5766 punch from 0x23da708 to 0x23e4f9c, (0xa894 bytes) 2024-09-16T13:00:51.715 INFO:teuthology.orchestra.run.smithi042.stdout:5767 writesame 0xdd944cd thru 0xdd962b0 data_size 0x2(0x1de4 bytes) 2024-09-16T13:00:51.720 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:51.712+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:51.732 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:51.732 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:51.732 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:51.732 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:51.734 INFO:teuthology.orchestra.run.smithi042.stdout:5769 trunc from 0xdd962b1 to 0x15fc536 2024-09-16T13:00:51.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:51.781 INFO:teuthology.orchestra.run.smithi042.stdout:5770 write 0x8313a20 thru 0x8313ddf (0x3c0 bytes) 2024-09-16T13:00:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:5772 write 0xc15c810 thru 0xc16765b (0xae4c bytes) 2024-09-16T13:00:51.821 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:51.812+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:51.821 INFO:teuthology.orchestra.run.smithi042.stdout:5773 read 0x539687f thru 0x53a2a36 (0xc1b8 bytes) 2024-09-16T13:00:51.822 INFO:teuthology.orchestra.run.smithi042.stdout:5775 write 0x3d973e6 thru 0x3da4ef8 (0xdb13 bytes) 2024-09-16T13:00:51.833 INFO:teuthology.orchestra.run.smithi042.stdout:5777 write 0xb8acdd2 thru 0xb8bb84f (0xea7e bytes) 2024-09-16T13:00:51.843 INFO:teuthology.orchestra.run.smithi042.stdout:5779 write 0x2eee48c thru 0x2ef10a7 (0x2c1c bytes) 2024-09-16T13:00:51.849 INFO:teuthology.orchestra.run.smithi042.stdout:5781 writesame 0xb66c922 thru 0xb67319d data_size 0x12(0x687c bytes) 2024-09-16T13:00:51.854 INFO:teuthology.orchestra.run.smithi042.stdout:5785 writesame 0x3eeadd4 thru 0x3ef5222 data_size 0x7(0xa44f bytes) 2024-09-16T13:00:51.867 INFO:teuthology.orchestra.run.smithi042.stdout:5786 read 0x8ff73ae thru 0x8ffe6c1 (0x7314 bytes) 2024-09-16T13:00:51.874 INFO:teuthology.orchestra.run.smithi042.stdout:5787 trunc from 0xc16765c to 0x87c9fb8 2024-09-16T13:00:51.893 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:51.893 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:51.893 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:51.893 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:51.893 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:51.898 INFO:teuthology.orchestra.run.smithi042.stdout:5790 write 0x6cf5594 thru 0x6cf5a53 (0x4c0 bytes) 2024-09-16T13:00:51.904 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:51.906 INFO:teuthology.orchestra.run.smithi042.stdout:5791 read 0x4082b53 thru 0x4084a98 (0x1f46 bytes) 2024-09-16T13:00:51.909 INFO:teuthology.orchestra.run.smithi042.stdout:5792 read 0x3d7951a thru 0x3d85c5b (0xc742 bytes) 2024-09-16T13:00:51.909 INFO:teuthology.orchestra.run.smithi042.stdout:5795 write 0x52701a1 thru 0x52788c7 (0x8727 bytes) 2024-09-16T13:00:51.920 INFO:teuthology.orchestra.run.smithi042.stdout:5796 write 0x249afbc thru 0x24a0273 (0x52b8 bytes) 2024-09-16T13:00:51.921 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:51.916+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:51.929 INFO:teuthology.orchestra.run.smithi042.stdout:5797 read 0x2837c48 thru 0x283c9c1 (0x4d7a bytes) 2024-09-16T13:00:51.943 INFO:teuthology.orchestra.run.smithi042.stdout:5798 write 0xad7501f thru 0xad7b155 (0x6137 bytes) 2024-09-16T13:00:51.951 INFO:teuthology.orchestra.run.smithi042.stdout:5799 punch from 0x8c5e719 to 0x8c611fc, (0x2ae3 bytes) 2024-09-16T13:00:51.951 INFO:teuthology.orchestra.run.smithi042.stdout:5800 write 0x189d8ee thru 0x18a088f (0x2fa2 bytes) 2024-09-16T13:00:51.956 INFO:teuthology.orchestra.run.smithi042.stdout:5801 write 0x9014ab7 thru 0x90245cc (0xfb16 bytes) 2024-09-16T13:00:51.970 INFO:teuthology.orchestra.run.smithi042.stdout:5802 read 0xa48a5e2 thru 0xa4900ea (0x5b09 bytes) 2024-09-16T13:00:51.970 INFO:teuthology.orchestra.run.smithi042.stdout:5803 read 0x4c65a22 thru 0x4c66a45 (0x1024 bytes) 2024-09-16T13:00:51.985 INFO:teuthology.orchestra.run.smithi042.stdout:5804 trunc from 0xad7b156 to 0x34865a8 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.947779904000001, 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:00:52.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:00:52.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.947727485000001, 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.027 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.947710637, 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:00:52.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:00:52.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.947694522999999, 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:00:52.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.039 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.947629592999998, 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:00:52.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:00:52.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.947613493999999, 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:00:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.947539713000001, 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:00:52.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:52.053 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.947522511999999, 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:00:52.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.947505808999999, 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:00:52.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.411194554000001, 2024-09-16T13:00:52.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:00:52.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.411177418000001, 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.411157836000001, 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.075 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.411126976999999, 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:00:52.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.411058880999999, 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:00:52.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:00:52.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:00:52.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159502 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4493312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612957+0000", 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.403771191000001, 2024-09-16T13:00:52.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070340508999999996, 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159502 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612957+0000", 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612957+0000", 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612958+0000", 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612960+0000", 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3750000000000001e-06 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612967+0000", 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3929999999999996e-06 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612970+0000", 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.23e-06 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683111+0000", 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070141677999999999 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683188+0000", 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6157999999999998e-05 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683298+0000", 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110221 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.089 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.403755025999999, 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:00:52.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.403737437, 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.098 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.403722544000001, 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:00:52.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:00:52.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:00:52.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:52.103 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.403690927, 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:00:52.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.107 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.403672906000001, 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:00:52.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:00:52.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:52.112 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:52.113 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:52.017+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:52.113 INFO:teuthology.orchestra.run.smithi042.stdout:5805 trunc from 0x34865a8 to 0xb6cfd56 2024-09-16T13:00:52.114 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:52.115 INFO:teuthology.orchestra.run.smithi042.stdout:5807 writesame 0x15a48ad thru 0x15a59cf data_size 0x1(0x1123 bytes) 2024-09-16T13:00:52.115 INFO:teuthology.orchestra.run.smithi042.stdout:5808 writesame 0x80d4bad thru 0x80da68a data_size 0x3(0x5ade bytes) 2024-09-16T13:00:52.115 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.115 INFO:teuthology.orchestra.run.smithi042.stdout:5810 writesame 0x221a1f6 thru 0x2225dfb data_size 0x29(0xbc06 bytes) 2024-09-16T13:00:52.115 INFO:teuthology.orchestra.run.smithi042.stdout:5811 writesame 0xbc3df14 thru 0xbc3f552 data_size 0x11(0x163f bytes) 2024-09-16T13:00:52.115 INFO:teuthology.orchestra.run.smithi042.stdout:5812 writesame 0x833333b thru 0x83360e1 data_size 0x1(0x2da7 bytes) 2024-09-16T13:00:52.116 INFO:teuthology.orchestra.run.smithi042.stdout:5813 write 0x85aa4d4 thru 0x85b91de (0xed0b bytes) 2024-09-16T13:00:52.116 INFO:teuthology.orchestra.run.smithi042.stdout:5815 write 0xa47b951 thru 0xa482b8d (0x723d bytes) 2024-09-16T13:00:52.116 INFO:teuthology.orchestra.run.smithi042.stdout:5816 writesame 0x822655e thru 0x822b6ae data_size 0x1b(0x5151 bytes) 2024-09-16T13:00:52.120 INFO:teuthology.orchestra.run.smithi042.stdout:5817 write 0x1bf8a52 thru 0x1c0667d (0xdc2c bytes) 2024-09-16T13:00:52.123 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:52.116+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:52.126 INFO:teuthology.orchestra.run.smithi042.stdout:5818 read 0x9af127e thru 0x9b00ee6 (0xfc69 bytes) 2024-09-16T13:00:52.126 INFO:teuthology.orchestra.run.smithi042.stdout:5819 read 0x5039f46 thru 0x503b77d (0x1838 bytes) 2024-09-16T13:00:52.127 INFO:teuthology.orchestra.run.smithi042.stdout:5823 writesame 0x8ca0a9e thru 0x8cad065 data_size 0x4(0xc5c8 bytes) 2024-09-16T13:00:52.146 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.146 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.146 INFO:teuthology.orchestra.run.smithi042.stdout:5826 write 0x2a74ef7 thru 0x2a82961 (0xda6b bytes) 2024-09-16T13:00:52.158 INFO:teuthology.orchestra.run.smithi042.stdout:5828 read 0x67f0407 thru 0x67f6f10 (0x6b0a bytes) 2024-09-16T13:00:52.171 INFO:teuthology.orchestra.run.smithi042.stdout:5830 write 0xc3e3842 thru 0xc3ecc38 (0x93f7 bytes) 2024-09-16T13:00:52.184 INFO:teuthology.orchestra.run.smithi042.stdout:5831 write 0x6eda5f thru 0x6f275c (0x4cfe bytes) 2024-09-16T13:00:52.187 INFO:teuthology.orchestra.run.smithi042.stdout:5832 writesame 0xaf8a2d5 thru 0xaf93c12 data_size 0x5(0x993e bytes) 2024-09-16T13:00:52.214 INFO:teuthology.orchestra.run.smithi042.stdout:5833 trunc from 0xc3ecc39 to 0x6380efb 2024-09-16T13:00:52.224 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:52.216+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:52.229 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:274975 4.8 4:10c0f49a:::rbd_object_map.11b5b95ccab:head [call lock.assert_locked in=44b,call rbd.object_map_update in=39b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:52.227379+0000", 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.00051106100000000005, 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054318200000000002, 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for sub ops", 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 274975 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:52.227379+0000", 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:52.227379+0000", 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:52.227380+0000", 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.113e-06 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:52.227387+0000", 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6560000000000003e-06 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:52.227400+0000", 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3135e-05 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:52.227406+0000", 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8599999999999998e-06 2024-09-16T13:00:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:52.227441+0000", 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4817999999999998e-05 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:52.227501+0000", 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9864000000000001e-05 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 7", 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:52.227746+0000", 2024-09-16T13:00:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000245231 2024-09-16T13:00:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-16T13:00:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 1 2024-09-16T13:00:52.234 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:5834 punch from 0x15645c0 to 0x156c620, (0x8060 bytes) 2024-09-16T13:00:52.239 INFO:teuthology.orchestra.run.smithi042.stdout:5836 writesame 0x11ac893 thru 0x11adf7e data_size 0x12(0x16ec bytes) 2024-09-16T13:00:52.241 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:52.241 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.242 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.271 INFO:teuthology.orchestra.run.smithi042.stdout:5841 writesame 0xe643560 thru 0xe6439e9 data_size 0x7(0x48a bytes) 2024-09-16T13:00:52.271 INFO:teuthology.orchestra.run.smithi042.stdout:5844 write 0x592cd28 thru 0x593bb92 (0xee6b bytes) 2024-09-16T13:00:52.287 INFO:teuthology.orchestra.run.smithi042.stdout:5845 read 0x67d13fa thru 0x67d22e3 (0xeea bytes) 2024-09-16T13:00:52.287 INFO:teuthology.orchestra.run.smithi042.stdout:5847 punch from 0x85afde7 to 0x85bf72c, (0xf945 bytes) 2024-09-16T13:00:52.287 INFO:teuthology.orchestra.run.smithi042.stdout:5849 write 0x10ec968 thru 0x10f49e9 (0x8082 bytes) 2024-09-16T13:00:52.319 INFO:teuthology.orchestra.run.smithi042.stdout:5850 trunc from 0xe6439ea to 0x1205fb8 2024-09-16T13:00:52.325 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:52.321+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:52.347 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:52.347 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:52.347 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:52.347 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:52.347 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:52.357 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:52.386 INFO:teuthology.orchestra.run.smithi042.stdout:5851 write 0x59b6ea0 thru 0x59b8b5f (0x1cc0 bytes) 2024-09-16T13:00:52.396 INFO:teuthology.orchestra.run.smithi042.stdout:5852 write 0x3fece78 thru 0x3ffc081 (0xf20a bytes) 2024-09-16T13:00:52.409 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.423 INFO:teuthology.orchestra.run.smithi042.stdout:5855 write 0x69fd75f thru 0x69ff006 (0x18a8 bytes) 2024-09-16T13:00:52.425 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:52.420+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:52.451 INFO:teuthology.orchestra.run.smithi042.stdout:5856 writesame 0xb675f11 thru 0xb67911b data_size 0x1(0x320b bytes) 2024-09-16T13:00:52.451 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.451 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.452 INFO:teuthology.orchestra.run.smithi042.stdout:5861 write 0x631dac1 thru 0x631f54b (0x1a8b bytes) 2024-09-16T13:00:52.457 INFO:teuthology.orchestra.run.smithi042.stdout:5862 write 0x360e2f8 thru 0x36186a8 (0xa3b1 bytes) 2024-09-16T13:00:52.465 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240664 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 152576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.91983585, 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047790817999999999, 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.466 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240664 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:00:52.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1900000000000003e-07 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543064+0000", 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4449999999999999e-06 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543079+0000", 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5221e-05 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543084+0000", 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8590000000000004e-06 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590750+0000", 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047665710999999999 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590801+0000", 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1827999999999999e-05 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590852+0000", 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0235000000000003e-05 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.470 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240665 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.919806446999999, 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047896192999999997, 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240665 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543091+0000", 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543094+0000", 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7149999999999998e-06 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543102+0000", 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6549999999999993e-06 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543108+0000", 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4840000000000003e-06 2024-09-16T13:00:52.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590885+0000", 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047777396 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590936+0000", 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0611e-05 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590987+0000", 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0785000000000003e-05 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240666 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 162816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.919784179000001, 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048010069000000002, 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.475 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240666 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:00:52.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8500000000000002e-07 2024-09-16T13:00:52.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543116+0000", 2024-09-16T13:00:52.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.531e-06 2024-09-16T13:00:52.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543123+0000", 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1069999999999999e-06 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543127+0000", 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4309999999999996e-06 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591020+0000", 2024-09-16T13:00:52.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047892413000000002 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591071+0000", 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1650000000000002e-05 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591123+0000", 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1452000000000001e-05 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240667 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.919765397000001, 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048125951, 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240667 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.480 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543132+0000", 2024-09-16T13:00:52.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5099999999999999e-07 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543134+0000", 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1500000000000002e-06 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543141+0000", 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543146+0000", 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9200000000000003e-06 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591158+0000", 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048011246 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591207+0000", 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9295999999999997e-05 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591257+0000", 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0426000000000001e-05 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240668 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 173056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.919745691999999, 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048243280999999999, 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.484 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240668 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543152+0000", 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543154+0000", 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5380000000000001e-06 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543161+0000", 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7920000000000004e-06 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543172+0000", 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1327e-05 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591290+0000", 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048118181000000003 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591344+0000", 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4061000000000003e-05 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591394+0000", 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9866e-05 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.489 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240766 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.489 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:00:52.489 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.918541367, 2024-09-16T13:00:52.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047716644000000003, 2024-09-16T13:00:52.489 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.489 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240766 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544356+0000", 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2500000000000005e-07 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544373+0000", 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6685999999999999e-05 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544381+0000", 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7190000000000001e-06 2024-09-16T13:00:52.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544384+0000", 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591969+0000", 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047585483999999997 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592021+0000", 2024-09-16T13:00:52.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1839000000000003e-05 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592072+0000", 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754000000000002e-05 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240767 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 201728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:00:52.493 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.918509492, 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047822479000000001, 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240767 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544388+0000", 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.51e-07 2024-09-16T13:00:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544390+0000", 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.824e-06 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544393+0000", 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.681e-06 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544396+0000", 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4940000000000002e-06 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592105+0000", 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047709184000000002 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592158+0000", 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2772999999999998e-05 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592210+0000", 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2071999999999999e-05 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240768 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.918496691, 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047944647, 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.498 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240768 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0400000000000002e-07 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544403+0000", 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1430000000000001e-06 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544406+0000", 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7450000000000001e-06 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544409+0000", 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592241+0000", 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047832146999999998 2024-09-16T13:00:52.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592291+0000", 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9289000000000002e-05 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592345+0000", 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4067000000000003e-05 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240769 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.918485066000001, 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048066225999999997, 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240769 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1399999999999998e-07 2024-09-16T13:00:52.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544413+0000", 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.42e-06 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544417+0000", 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.529e-06 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544420+0000", 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5620000000000002e-06 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592378+0000", 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047958354000000002 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592427+0000", 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9013e-05 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592478+0000", 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1034e-05 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240770 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.918474803000001, 2024-09-16T13:00:52.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048189432999999997, 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240770 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3200000000000001e-07 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544424+0000", 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4449999999999999e-06 2024-09-16T13:00:52.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544427+0000", 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5049999999999999e-06 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544430+0000", 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.61e-06 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592511+0000", 2024-09-16T13:00:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048080912000000003 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592563+0000", 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2107000000000001e-05 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592611+0000", 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8522000000000001e-05 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.511 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240771 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.918440196000001, 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290128000000002, 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240771 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5700000000000002e-07 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544459+0000", 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6840000000000001e-06 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544466+0000", 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999997e-06 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544469+0000", 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592645+0000", 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048175216999999999 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592696+0000", 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1292999999999998e-05 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592747+0000", 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0751000000000002e-05 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.515 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240772 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.918416901000001, 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048403333999999999, 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240772 2024-09-16T13:00:52.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544482+0000", 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8390000000000001e-06 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544486+0000", 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9369999999999997e-06 2024-09-16T13:00:52.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544489+0000", 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8459999999999999e-06 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592779+0000", 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290301000000001 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592831+0000", 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2335e-05 2024-09-16T13:00:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592883+0000", 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1786999999999997e-05 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240773 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.918405516, 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048527963, 2024-09-16T13:00:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240773 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:00:52.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544492+0000", 2024-09-16T13:00:52.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.84e-07 2024-09-16T13:00:52.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544493+0000", 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6279999999999999e-06 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544498+0000", 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4689999999999999e-06 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544500+0000", 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592916+0000", 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048415672 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592965+0000", 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9120999999999999e-05 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593019+0000", 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3950999999999999e-05 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240774 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.918393861, 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048649546000000002, 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240774 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544505+0000", 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.61e-06 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544508+0000", 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4929999999999998e-06 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544511+0000", 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4169999999999999e-06 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593051+0000", 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048539604 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593103+0000", 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2068999999999999e-05 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593152+0000", 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9870999999999997e-05 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240775 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.918383606999999, 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048773328999999997, 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240775 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:00:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545618+0000", 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011045810000000001 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545625+0000", 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6079999999999998e-06 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545644+0000", 2024-09-16T13:00:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.823e-05 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545653+0000", 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3000000000000007e-06 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593186+0000", 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047533387000000003 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593235+0000", 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8767000000000003e-05 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593286+0000", 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1456000000000003e-05 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240776 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:00:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.917231469000001, 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047754889000000002, 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240776 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545666+0000", 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-06 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545671+0000", 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2270000000000004e-06 2024-09-16T13:00:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545681+0000", 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0506000000000001e-05 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545687+0000", 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0970000000000002e-06 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593320+0000", 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047632999000000002 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593369+0000", 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8882999999999999e-05 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593420+0000", 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1127e-05 2024-09-16T13:00:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240777 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.917202848000001, 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047878318000000003, 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240777 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545695+0000", 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545698+0000", 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0709999999999999e-06 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545707+0000", 2024-09-16T13:00:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7629999999999995e-06 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545713+0000", 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2839999999999997e-06 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593453+0000", 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047740281000000002 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593518+0000", 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5493000000000005e-05 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593572+0000", 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3739000000000002e-05 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240778 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.917178409, 2024-09-16T13:00:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047999411999999998, 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240778 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545719+0000", 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.13e-07 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545728+0000", 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1560000000000001e-06 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545901+0000", 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017311599999999999 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545909+0000", 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.028e-06 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593606+0000", 2024-09-16T13:00:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047696660000000002 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593666+0000", 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9611999999999998e-05 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593718+0000", 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2227e-05 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240951 4.8 4:10cc0d5f:::rbd_data.11b552946c0f.0000000000000120:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.909098139999999, 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048226894999999999, 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240951 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2300000000000002e-07 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553801+0000", 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.686e-06 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553805+0000", 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3039999999999998e-06 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553809+0000", 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2024-09-16T13:00:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.599994+0000", 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046185154999999999 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.600112+0000", 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118589 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.602026+0000", 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019131370000000001 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:272626 4.12 4:4935e950:::rbd_data.11b59088eb24.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0050705180000001, 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022439990000001, 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 272626 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457828+0000", 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457835+0000", 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7900000000000002e-06 2024-09-16T13:00:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457853+0000", 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7671000000000001e-05 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457859+0000", 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2949999999999999e-06 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457913+0000", 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4112e-05 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457922+0000", 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0850000000000003e-06 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.457879+0000", 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.99995641700000004 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458079+0000", 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020072300000000001 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458304+0000", 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000224601 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459393+0000", 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010886170000000001 2024-09-16T13:00:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459877+0000", 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048443500000000002 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460006+0000", 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012914 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460070+0000", 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4074999999999999e-05 2024-09-16T13:00:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.559 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.559 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.559 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.559 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.559 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:52.559 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T13:00:52.561 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.561 INFO:teuthology.orchestra.run.smithi042.stdout:5864 write 0xb9e4882 thru 0xb9f439e (0xfb1d bytes) 2024-09-16T13:00:52.561 INFO:teuthology.orchestra.run.smithi042.stdout:5865 write 0x8d6421b thru 0x8d6f4b4 (0xb29a bytes) 2024-09-16T13:00:52.561 INFO:teuthology.orchestra.run.smithi042.stdout:5866 write 0x8071a3f thru 0x8076ebc (0x547e bytes) 2024-09-16T13:00:52.561 INFO:teuthology.orchestra.run.smithi042.stdout:5867 trunc from 0xb9f439f to 0x2ecbaa3 2024-09-16T13:00:52.562 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:52.562 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:52.557+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:52.565 INFO:teuthology.orchestra.run.smithi042.stdout:5868 write 0x8202140 thru 0x820427d (0x213e bytes) 2024-09-16T13:00:52.577 INFO:teuthology.orchestra.run.smithi042.stdout:5871 writesame 0x1b6ee77 thru 0x1b7c28a data_size 0x62(0xd414 bytes) 2024-09-16T13:00:52.588 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:5873 writesame 0xab59629 thru 0xab61737 data_size 0x3(0x810f bytes) 2024-09-16T13:00:52.608 INFO:teuthology.orchestra.run.smithi042.stdout:5874 trunc from 0xab61738 to 0x96a4aa1 2024-09-16T13:00:52.618 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.619 INFO:teuthology.orchestra.run.smithi042.stdout:5876 read 0x96667c4 thru 0x96750e2 (0xe91f bytes) 2024-09-16T13:00:52.619 INFO:teuthology.orchestra.run.smithi042.stdout:5881 read 0x117ecf3 thru 0x1185f27 (0x7235 bytes) 2024-09-16T13:00:52.625 INFO:teuthology.orchestra.run.smithi042.stdout:5882 trunc from 0x96a4aa1 to 0xc46a768 2024-09-16T13:00:52.636 INFO:teuthology.orchestra.run.smithi042.stdout:5883 write 0xbb7e9a3 thru 0xbb8952d (0xab8b bytes) 2024-09-16T13:00:52.653 INFO:teuthology.orchestra.run.smithi042.stdout:5884 writesame 0x74a9822 thru 0x74ae166 data_size 0x1(0x4945 bytes) 2024-09-16T13:00:52.661 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:52.656+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:52.668 INFO:teuthology.orchestra.run.smithi042.stdout:5885 write 0xc0d4301 thru 0xc0df9f3 (0xb6f3 bytes) 2024-09-16T13:00:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:5890 punch from 0x33fb3ae to 0x33fc8de, (0x1530 bytes) 2024-09-16T13:00:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:5891 read 0x1b04a78 thru 0x1b09a28 (0x4fb1 bytes) 2024-09-16T13:00:52.676 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:52.676 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:52.676 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:52.676 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:52.683 INFO:teuthology.orchestra.run.smithi042.stdout:5892 trunc from 0xc46a768 to 0x65dd6c1 2024-09-16T13:00:52.685 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:52.727 INFO:teuthology.orchestra.run.smithi042.stdout:5893 writesame 0xe7c0c22 thru 0xe7ca9a5 data_size 0x22(0x9d84 bytes) 2024-09-16T13:00:52.727 INFO:teuthology.orchestra.run.smithi042.stdout:5894 write 0x67988c6 thru 0x679ff10 (0x764b bytes) 2024-09-16T13:00:52.736 INFO:teuthology.orchestra.run.smithi042.stdout:5895 read 0x46b9c26 thru 0x46c3a0e (0x9de9 bytes) 2024-09-16T13:00:52.736 INFO:teuthology.orchestra.run.smithi042.stdout:5897 write 0x5606a7 thru 0x56a3b3 (0x9d0d bytes) 2024-09-16T13:00:52.740 INFO:teuthology.orchestra.run.smithi042.stdout:5898 read 0xcf987f7 thru 0xcfa4c62 (0xc46c bytes) 2024-09-16T13:00:52.740 INFO:teuthology.orchestra.run.smithi042.stdout:5900 write 0xd90830e thru 0xd90ecfc (0x69ef bytes) 2024-09-16T13:00:52.748 INFO:teuthology.orchestra.run.smithi042.stdout:5901 punch from 0x7b92aa9 to 0x7b95236, (0x278d bytes) 2024-09-16T13:00:52.748 INFO:teuthology.orchestra.run.smithi042.stdout:5902 read 0xdc154ed thru 0xdc16923 (0x1437 bytes) 2024-09-16T13:00:52.748 INFO:teuthology.orchestra.run.smithi042.stdout:5903 writesame 0xdd9df81 thru 0xdda0c5c data_size 0x16(0x2cdc bytes) 2024-09-16T13:00:52.755 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.755 INFO:teuthology.orchestra.run.smithi042.stdout:5905 punch from 0x5510d23 to 0x551e329, (0xd606 bytes) 2024-09-16T13:00:52.755 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.755 INFO:teuthology.orchestra.run.smithi042.stdout:5907 write 0x397bc9b thru 0x39810fa (0x5460 bytes) 2024-09-16T13:00:52.762 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:52.761+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:52.766 INFO:teuthology.orchestra.run.smithi042.stdout:5908 write 0x74368f9 thru 0x74430e6 (0xc7ee bytes) 2024-09-16T13:00:52.776 INFO:teuthology.orchestra.run.smithi042.stdout:5909 write 0xb2612da thru 0xb26f1aa (0xded1 bytes) 2024-09-16T13:00:52.787 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.787 INFO:teuthology.orchestra.run.smithi042.stdout:5911 read 0x26e80af thru 0x26f5112 (0xd064 bytes) 2024-09-16T13:00:52.787 INFO:teuthology.orchestra.run.smithi042.stdout:5912 read 0x369a13d thru 0x36a1729 (0x75ed bytes) 2024-09-16T13:00:52.787 INFO:teuthology.orchestra.run.smithi042.stdout:5914 punch from 0x661e385 to 0x6629b1f, (0xb79a bytes) 2024-09-16T13:00:52.787 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.787 INFO:teuthology.orchestra.run.smithi042.stdout:5918 writesame 0xbec49ce thru 0xbec868e data_size 0x1(0x3cc1 bytes) 2024-09-16T13:00:52.793 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:52.794 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:52.794 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:52.794 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:52.794 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:52.803 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:52.804 INFO:teuthology.orchestra.run.smithi042.stdout:5919 writesame 0xcf2bf5f thru 0xcf3aeda data_size 0x34(0xef7c bytes) 2024-09-16T13:00:52.819 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.819 INFO:teuthology.orchestra.run.smithi042.stdout:5922 write 0x12956fe thru 0x12a25f9 (0xcefc bytes) 2024-09-16T13:00:52.822 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.822 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.822 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.822 INFO:teuthology.orchestra.run.smithi042.stdout:5927 write 0x3339003 thru 0x33473e0 (0xe3de bytes) 2024-09-16T13:00:52.833 INFO:teuthology.orchestra.run.smithi042.stdout:5931 write 0x82d1613 thru 0x82db38d (0x9d7b bytes) 2024-09-16T13:00:52.843 INFO:teuthology.orchestra.run.smithi042.stdout:5932 read 0x328d574 thru 0x328ee71 (0x18fe bytes) 2024-09-16T13:00:52.844 INFO:teuthology.orchestra.run.smithi042.stdout:5933 punch from 0xc184e95 to 0xc192f59, (0xe0c4 bytes) 2024-09-16T13:00:52.863 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:52.861+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:52.863 INFO:teuthology.orchestra.run.smithi042.stdout:5934 trunc from 0xe7ca9a6 to 0x298773b 2024-09-16T13:00:52.892 INFO:teuthology.orchestra.run.smithi042.stdout:5935 read 0x18562a thru 0x18a11a (0x4af1 bytes) 2024-09-16T13:00:52.892 INFO:teuthology.orchestra.run.smithi042.stdout:5936 punch from 0x1c393af to 0x1c442fb, (0xaf4c bytes) 2024-09-16T13:00:52.892 INFO:teuthology.orchestra.run.smithi042.stdout:5938 read 0xbc57e8 thru 0xbd2aac (0xd2c5 bytes) 2024-09-16T13:00:52.893 INFO:teuthology.orchestra.run.smithi042.stdout:5939 writesame 0x241ca2b thru 0x24292c1 data_size 0x1(0xc897 bytes) 2024-09-16T13:00:52.918 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:52.920 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:52.920 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:52.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 85.552247647000002, 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T13:00:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T13:00:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:18031 4.4 4:2649aabf:::rbd_data.11b51a3eb68b.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~5304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.755924413999999, 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94984657100000003, 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 18031 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160519+0000", 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3029999999999998e-06 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160536+0000", 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7181999999999999e-05 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160564+0000", 2024-09-16T13:00:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971e-05 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160576+0000", 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160669+0000", 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3282999999999994e-05 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160680+0000", 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107188+0000", 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94650794999999999 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107644+0000", 2024-09-16T13:00:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00045604499999999997 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.108257+0000", 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061324700000000003 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.109811+0000", 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015538749999999999 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110175+0000", 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036419399999999998 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110285+0000", 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010938900000000001 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110364+0000", 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8689000000000005e-05 2024-09-16T13:00:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:64609 4.e 4:7394d9c8:::rbd_object_map.11b5c2ee3542:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.654715250000002, 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.020058463, 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 64609 2024-09-16T13:00:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261729+0000", 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4430000000000002e-06 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261741+0000", 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261780+0000", 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8722e-05 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261798+0000", 2024-09-16T13:00:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8150000000000001e-05 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261927+0000", 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000129243 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261948+0000", 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1047e-05 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273712+0000", 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011763704999999999 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273720+0000", 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4309999999999994e-06 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280059+0000", 2024-09-16T13:00:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.006338744 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280297+0000", 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000237667 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281645+0000", 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001348555 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281702+0000", 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6342999999999999e-05 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281785+0000", 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2782999999999997e-05 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254540 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 176128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816406+0000", 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1000350260000005, 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053609236999999997, 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254540 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816406+0000", 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816406+0000", 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816407+0000", 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6000000000000004e-07 2024-09-16T13:00:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816409+0000", 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1349999999999999e-06 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816415+0000", 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9780000000000002e-06 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816424+0000", 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0990000000000005e-06 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.869803+0000", 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053378429999999998 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.869904+0000", 2024-09-16T13:00:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101937 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870016+0000", 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111098 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254541 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 183296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816431+0000", 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1000099920000004, 2024-09-16T13:00:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053814215999999998, 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254541 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816431+0000", 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816431+0000", 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816432+0000", 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816441+0000", 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8550000000000008e-06 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816457+0000", 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5509e-05 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816463+0000", 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8800000000000002e-06 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870060+0000", 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053596442000000001 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870142+0000", 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2232999999999997e-05 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870246+0000", 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103509 2024-09-16T13:00:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254542 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 186368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816469+0000", 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0999719520000006, 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054024784999999999, 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254542 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816469+0000", 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816469+0000", 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816470+0000", 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1999999999999998e-07 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816472+0000", 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.255e-06 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816479+0000", 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2539999999999998e-06 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816484+0000", 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.062e-06 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870290+0000", 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053806132 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870379+0000", 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9128000000000005e-05 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870494+0000", 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115134 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254543 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 193536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0999535730000005, 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054211546999999999, 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254543 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:00:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2e-07 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816490+0000", 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1220000000000002e-06 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816510+0000", 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9457999999999999e-05 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816522+0000", 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2585000000000001e-05 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870529+0000", 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054006981000000003 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870600+0000", 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0073999999999993e-05 2024-09-16T13:00:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870699+0000", 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9807000000000002e-05 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254544 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 196608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816527+0000", 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0999143920000005, 2024-09-16T13:00:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054393364999999999, 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254544 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816527+0000", 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816527+0000", 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816528+0000", 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-07 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816530+0000", 2024-09-16T13:00:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1619999999999998e-06 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816542+0000", 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2459e-05 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816555+0000", 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2532e-05 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870738+0000", 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054183505 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870818+0000", 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9425999999999995e-05 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870920+0000", 2024-09-16T13:00:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010262700000000001 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254545 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 203776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816559+0000", 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0998818890000006, 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054590611999999997, 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254545 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816559+0000", 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816559+0000", 2024-09-16T13:00:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816560+0000", 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9699999999999996e-07 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816562+0000", 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3549999999999999e-06 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816569+0000", 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3879999999999997e-06 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816573+0000", 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0720000000000001e-06 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870958+0000", 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054385019 2024-09-16T13:00:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871029+0000", 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1291000000000006e-05 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871150+0000", 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012089 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254546 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 206848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816577+0000", 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0998643950000009, 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054799678999999997, 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254546 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816577+0000", 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816577+0000", 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816578+0000", 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6499999999999999e-07 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816580+0000", 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1270000000000001e-06 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816591+0000", 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1838000000000001e-05 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816595+0000", 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9349999999999996e-06 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871208+0000", 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054612978 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871275+0000", 2024-09-16T13:00:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6774999999999997e-05 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871377+0000", 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101461 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254547 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 214016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816600+0000", 2024-09-16T13:00:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0998411580000003, 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054976553999999997, 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254547 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816600+0000", 2024-09-16T13:00:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816600+0000", 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816601+0000", 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7000000000000005e-07 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816603+0000", 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2500000000000001e-06 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816609+0000", 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9379999999999997e-06 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816615+0000", 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3860000000000004e-06 2024-09-16T13:00:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871395+0000", 2024-09-16T13:00:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054779732999999997 2024-09-16T13:00:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871446+0000", 2024-09-16T13:00:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0642000000000001e-05 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871577+0000", 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013103499999999999 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254548 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 217088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816619+0000", 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0998218939999997, 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055196303000000002, 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254548 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816619+0000", 2024-09-16T13:00:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816619+0000", 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816620+0000", 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2799999999999996e-07 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816622+0000", 2024-09-16T13:00:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9999999999999999e-06 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816637+0000", 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4922e-05 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816641+0000", 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1269999999999996e-06 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871623+0000", 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054981587999999998 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871688+0000", 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5148999999999996e-05 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871816+0000", 2024-09-16T13:00:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012798900000000001 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254549 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 224256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0997658250000004, 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055369889999999998, 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254549 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:00:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7499999999999999e-07 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816679+0000", 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6120000000000001e-06 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816686+0000", 2024-09-16T13:00:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816690+0000", 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0439999999999997e-06 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871868+0000", 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055177858000000003 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871933+0000", 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4857000000000006e-05 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872045+0000", 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112858 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254550 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 227328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:00:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0997475879999996, 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055577646000000001, 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254550 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:00:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:00:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:00:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5599999999999995e-07 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816696+0000", 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9439999999999999e-06 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816702+0000", 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7899999999999996e-06 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816707+0000", 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0520000000000001e-06 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872108+0000", 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055400479000000002 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872184+0000", 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6123000000000003e-05 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872271+0000", 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7701999999999995e-05 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254551 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 234496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816711+0000", 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0997302980000008, 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055772654999999997, 2024-09-16T13:00:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254551 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816711+0000", 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816711+0000", 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816712+0000", 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2e-07 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816714+0000", 2024-09-16T13:00:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0770000000000002e-06 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816720+0000", 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0750000000000003e-06 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816725+0000", 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0069999999999997e-06 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872306+0000", 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055580967000000002 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872386+0000", 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9956999999999994e-05 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872484+0000", 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8052000000000002e-05 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254552 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 237568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816729+0000", 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0997125230000009, 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055953530000000001, 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254552 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816729+0000", 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816729+0000", 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816730+0000", 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0999999999999997e-07 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816732+0000", 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9769999999999999e-06 2024-09-16T13:00:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816738+0000", 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0650000000000004e-06 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816743+0000", 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9969999999999998e-06 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872525+0000", 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055781981000000001 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872574+0000", 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9178000000000002e-05 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872682+0000", 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108522 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254553 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0996947390000003, 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056166305999999999, 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254553 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2e-07 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816754+0000", 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9820000000000002e-06 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816760+0000", 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816764+0000", 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8940000000000003e-06 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872739+0000", 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055974278000000002 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872798+0000", 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9564000000000001e-05 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872913+0000", 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114825 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254554 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 247808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816768+0000", 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0996731139999998, 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056329235999999998, 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254554 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816768+0000", 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816768+0000", 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816769+0000", 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-07 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816771+0000", 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.928e-06 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816777+0000", 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8819999999999998e-06 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816783+0000", 2024-09-16T13:00:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4819999999999999e-06 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872935+0000", 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056152226 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873003+0000", 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7581000000000002e-05 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873097+0000", 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4482999999999996e-05 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254555 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 254976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0996415850000005, 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056501814999999997, 2024-09-16T13:00:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254555 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8699999999999995e-07 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816803+0000", 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.255e-06 2024-09-16T13:00:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816809+0000", 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3119999999999999e-06 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816813+0000", 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.856e-06 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873130+0000", 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056317045000000003 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873192+0000", 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1857000000000001e-05 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873302+0000", 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010990299999999999 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254556 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 258048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0996245709999997, 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056686382, 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254556 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816819+0000", 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.046e-06 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816825+0000", 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8860000000000001e-06 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816829+0000", 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873320+0000", 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056491373999999997 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873383+0000", 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2312999999999998e-05 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873503+0000", 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012061 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.022 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:53.024 INFO:teuthology.orchestra.run.smithi042.stdout:5942 write 0xda27420 thru 0xda31ee3 (0xaac4 bytes) 2024-09-16T13:00:53.024 INFO:teuthology.orchestra.run.smithi042.stdout:5943 punch from 0xc0758e5 to 0xc07e9bd, (0x90d8 bytes) 2024-09-16T13:00:53.025 INFO:teuthology.orchestra.run.smithi042.stdout:5944 punch from 0x373a2fa to 0x373ca6f, (0x2775 bytes) 2024-09-16T13:00:53.025 INFO:teuthology.orchestra.run.smithi042.stdout:5945 punch from 0x72de837 to 0x72e3605, (0x4dce bytes) 2024-09-16T13:00:53.025 INFO:teuthology.orchestra.run.smithi042.stdout:5947 write 0x10e27a4 thru 0x10f1deb (0xf648 bytes) 2024-09-16T13:00:53.025 INFO:teuthology.orchestra.run.smithi042.stdout:5948 trunc from 0xda31ee4 to 0x65a0ea2 2024-09-16T13:00:53.025 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:53.026 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:53.016+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:53.028 INFO:teuthology.orchestra.run.smithi042.stdout:5949 write 0xe612000 thru 0xe613118 (0x1119 bytes) 2024-09-16T13:00:53.035 INFO:teuthology.orchestra.run.smithi042.stdout:5950 write 0x7983c5 thru 0x7a4126 (0xbd62 bytes) 2024-09-16T13:00:53.039 INFO:teuthology.orchestra.run.smithi042.stdout:5951 write 0x89bda96 thru 0x89c78f9 (0x9e64 bytes) 2024-09-16T13:00:53.064 INFO:teuthology.orchestra.run.smithi042.stdout:5952 write 0xeb3d3b5 thru 0xeb4af37 (0xdb83 bytes) 2024-09-16T13:00:53.081 INFO:teuthology.orchestra.run.smithi042.stdout:5953 read 0x8106310 thru 0x810f3bb (0x90ac bytes) 2024-09-16T13:00:53.082 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:53.082 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:53.082 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:53.083 INFO:teuthology.orchestra.run.smithi042.stdout:5957 write 0x3542ba7 thru 0x3552202 (0xf65c bytes) 2024-09-16T13:00:53.090 INFO:teuthology.orchestra.run.smithi042.stdout:5958 read 0x8bc8be5 thru 0x8bd03ed (0x7809 bytes) 2024-09-16T13:00:53.091 INFO:teuthology.orchestra.run.smithi042.stdout:5959 read 0x28b1982 thru 0x28b33e3 (0x1a62 bytes) 2024-09-16T13:00:53.091 INFO:teuthology.orchestra.run.smithi042.stdout:5960 write 0xe883a68 thru 0xe88800b (0x45a4 bytes) 2024-09-16T13:00:53.102 INFO:teuthology.orchestra.run.smithi042.stdout:5961 write 0x126075b thru 0x126e2e8 (0xdb8e bytes) 2024-09-16T13:00:53.112 INFO:teuthology.orchestra.run.smithi042.stdout:5962 write 0x8b20c00 thru 0x8b2f0d7 (0xe4d8 bytes) 2024-09-16T13:00:53.117 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:53.117 INFO:teuthology.orchestra.run.smithi042.stdout:5964 writesame 0xd679800 thru 0xd67c110 data_size 0x1(0x2911 bytes) 2024-09-16T13:00:53.125 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:53.116+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:53.131 INFO:teuthology.orchestra.run.smithi042.stdout:5965 clone 38 order 21 su 262144 sc 14 2024-09-16T13:00:53.169 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:53.169 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:53.169 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:53.169 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:53.178 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:53.225 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:53.221+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:53.276 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:53.277 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:53.277 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:53.277 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:53.277 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:53.285 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:53.326 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:53.324+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:53.428 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:53.420+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:53.435 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:53.435 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:53.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:53.435 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:53.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.435 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:53.435 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:53.435 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 127.88055083, 2024-09-16T13:00:53.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:00:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 127.055433783, 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:00:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 127.055386527, 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:00:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:00:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 127.055183591, 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:00:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:00:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:00:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 127.05515232899999, 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:00:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:00:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 127.055116567, 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:00:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:00:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 127.054921937, 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:00:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 127.054875092, 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:00:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:00:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:00:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:00:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 126.510598921, 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:00:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 126.510545014, 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:00:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:00:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:00:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:00:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:00:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:00:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 126.51050270899999, 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:00:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:00:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:00:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:00:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 126.510344284, 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:00:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 126.51029480299999, 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 126.510164233, 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:00:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:00:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:00:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 126.50998272, 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 125.69945505299999, 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:00:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 125.699306099, 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:00:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64507 4.1e e83/31 4:7f7c7834:::rbd_data.11b5c2ee3542.0000000000000008:head v 83'225, mlcod=81'223)", 2024-09-16T13:00:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.659892812000002, 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021284401000000001, 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:00:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770380+0000", 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8999999999999998e-07 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777141+0000", 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067617670000000001 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777157+0000", 2024-09-16T13:00:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777167+0000", 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777267+0000", 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100364 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777304+0000", 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7165000000000001e-05 2024-09-16T13:00:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791601+0000", 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014297275999999999 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791664+0000", 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2217000000000004e-05 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64509 4.15 e83/31 4:ad2e18f4:::rbd_data.11b5c2ee3542.0000000000000009:head v 83'183, mlcod=81'180)", 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.653134909999999, 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014054395000000001, 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777138+0000", 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778534+0000", 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013956769999999999 2024-09-16T13:00:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778555+0000", 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1046000000000001e-05 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778564+0000", 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9290000000000004e-06 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778633+0000", 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9363000000000007e-05 2024-09-16T13:00:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778666+0000", 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135000000000003e-05 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791145+0000", 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012479743 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791192+0000", 2024-09-16T13:00:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6156000000000003e-05 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:217754 4.17 e92/31 4:ee4048fa:::rbd_data.11b58362ce83.0000000000000002:head v 92'697, mlcod=91'693)", 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.930964855999999, 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011416822, 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499309+0000", 2024-09-16T13:00:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6929999999999999e-06 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499322+0000", 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499354+0000", 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135999999999999e-05 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499365+0000", 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111e-05 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499504+0000", 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013877599999999999 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499565+0000", 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1939000000000004e-05 2024-09-16T13:00:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510591+0000", 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011025874999999999 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510724+0000", 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013266299999999999 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:53.550 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:53.550 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:53.551 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T13:00:53.552 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:53.544+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:53.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:53.616 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:53.651 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:53.644+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:53.660 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone37 from 0xeb4af38 (overlap 0x5d6737) to 0x4161d9 2024-09-16T13:00:53.712 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:53.713 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:53.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:53.741 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:53.737+0000 7f466bedb540 -1 osd.4 83 log_to_monitors true 2024-09-16T13:00:53.761 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:53.761 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:53.756+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:53.784 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:53.803 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:53.822 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:53.842 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-16T13:00:53.850 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:53.853 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:53.848+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:53.877 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:53.891 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:53.900 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:53.900 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:53.901 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-16T13:00:53.901 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:53.901 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:53.901 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:53.904 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:00:53.904 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:00:53.956 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:53.948+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:54.046 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:54.047 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:54.055 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:54.053+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:54.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:54.156 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:54.153+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:54.157 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:54.157 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:54.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:54.258 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:54.253+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:54.265 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:54.265 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:54.275 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:54.315 INFO:teuthology.orchestra.run.smithi042.stdout:5966 read 0x40033ac thru 0x400ac01 (0x7856 bytes) 2024-09-16T13:00:54.316 INFO:teuthology.orchestra.run.smithi042.stdout:5967 read 0x3f364df thru 0x3f36913 (0x435 bytes) 2024-09-16T13:00:54.317 INFO:teuthology.orchestra.run.smithi042.stdout:5968 read 0xa580185 thru 0xa586167 (0x5fe3 bytes) 2024-09-16T13:00:54.318 INFO:teuthology.orchestra.run.smithi042.stdout:5969 read 0x7565fd4 thru 0x756dd02 (0x7d2f bytes) 2024-09-16T13:00:54.320 INFO:teuthology.orchestra.run.smithi042.stdout:5970 trunc from 0xeb4af38 to 0xd8ba11b 2024-09-16T13:00:54.359 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:54.352+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:54.372 INFO:teuthology.orchestra.run.smithi042.stdout:5973 write 0x13e5b7d thru 0x13f5051 (0xf4d5 bytes) 2024-09-16T13:00:54.392 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:54.392 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:54.392 INFO:teuthology.orchestra.run.smithi042.stdout:5974 trunc from 0xd8ba11b to 0x6996299 2024-09-16T13:00:54.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:54.436 INFO:teuthology.orchestra.run.smithi042.stdout:5976 punch from 0x1ff4f63 to 0x1ffcae4, (0x7b81 bytes) 2024-09-16T13:00:54.436 INFO:teuthology.orchestra.run.smithi042.stdout:5977 read 0xfd2c9 thru 0x1072d6 (0xa00e bytes) 2024-09-16T13:00:54.447 INFO:teuthology.orchestra.run.smithi042.stdout:5979 trunc from 0x6996299 to 0xc1d6484 2024-09-16T13:00:54.457 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:54.459 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:54.452+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:54.470 INFO:teuthology.orchestra.run.smithi042.stdout:5984 writesame 0xd2f827f thru 0xd300964 data_size 0x1(0x86e6 bytes) 2024-09-16T13:00:54.470 INFO:teuthology.orchestra.run.smithi042.stdout:5986 write 0xc5e6612 thru 0xc5ee42e (0x7e1d bytes) 2024-09-16T13:00:54.476 INFO:teuthology.orchestra.run.smithi042.stdout:5987 writesame 0x351705c thru 0x3525c5f data_size 0x72(0xec04 bytes) 2024-09-16T13:00:54.489 INFO:teuthology.orchestra.run.smithi042.stdout:5988 write 0x118af12 thru 0x118f41a (0x4509 bytes) 2024-09-16T13:00:54.512 INFO:teuthology.orchestra.run.smithi042.stdout:5989 write 0xcf3c99d thru 0xcf3eb87 (0x21eb bytes) 2024-09-16T13:00:54.517 INFO:teuthology.orchestra.run.smithi042.stdout:5992 writesame 0xe857d0 thru 0xe8c841 data_size 0x25(0x7072 bytes) 2024-09-16T13:00:54.529 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:54.525+0000 7f466041e700 -1 osd.4 107 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T13:00:54.536 INFO:teuthology.orchestra.run.smithi042.stdout:5994 read 0x2a95054 thru 0x2a9d182 (0x812f bytes) 2024-09-16T13:00:54.537 INFO:teuthology.orchestra.run.smithi042.stdout:5995 read 0xbab1e7b thru 0xbab9ed1 (0x8057 bytes) 2024-09-16T13:00:54.539 INFO:teuthology.orchestra.run.smithi042.stdout:5996 trunc from 0xd300965 to 0xb95eebc 2024-09-16T13:00:54.551 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:54.551 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:54.558 INFO:teuthology.orchestra.run.smithi042.stdout:5997 write 0x8dc74a3 thru 0x8dd546d (0xdfcb bytes) 2024-09-16T13:00:54.560 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:54.557+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:54.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:54.574 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:00:54.575 INFO:teuthology.orchestra.run.smithi042.stdout:5998 trunc from 0xb95eebc to 0x52f6645 2024-09-16T13:00:54.620 INFO:teuthology.orchestra.run.smithi042.stdout:5999 write 0x6fd462d thru 0x6fe2b06 (0xe4da bytes) 2024-09-16T13:00:54.626 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-16T13:00:54.661 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:54.657+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:54.676 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:54.676 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:54.684 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:54.684 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:54.685 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:54.685 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:54.686 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:54.694 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T13:00:54.708 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-16T13:00:54.762 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:54.756+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:54.787 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:54.787 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:54.797 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:54.863 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:54.856+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:54.903 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:54.903 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:54.915 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:54.963 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:54.956+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:55.011 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:55.012 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:55.022 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:55.064 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:55.061+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:00:55.112 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:55.112 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:55.121 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:55.165 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:55.160+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:55.228 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:55.229 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:55.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:55.266 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:55.264+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:55.333 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:55.333 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:55.342 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:55.367 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:55.360+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:55.440 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:55.441 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:55.449 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:55.468 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:55.460+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:55.517 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T13:00:55.529 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T13:00:55.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T13:00:55.559 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:55.559 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:55.569 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:55.573 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:55.568+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:55.663 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:55.663 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:55.670 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:55.664+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:55.672 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:55.771 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:55.764+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:55.773 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:55.773 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:55.783 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:55.871 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:55.864+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:55.880 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:55.880 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:55.890 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:55.905 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:00:55.972 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:55.964+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:55.992 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:55.992 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:56.001 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:56.045 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:56.070 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:56.073 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:56.068+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:56.091 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:56.104 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:56.104 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:56.109 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:56.115 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:00:56.130 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:00:56.150 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:56.171 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:56.174 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:56.169+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:56.191 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:56.194 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-16T13:00:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:56.195 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:56.209 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:56.209 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:00:56.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:56.209 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.67332804300000004, 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.67337001200000002, 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for peered", 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279128 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534648+0000", 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1459999999999999e-06 2024-09-16T13:00:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534655+0000", 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5220000000000004e-06 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534674+0000", 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9717000000000001e-05 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534683+0000", 2024-09-16T13:00:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534759+0000", 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6074000000000004e-05 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534770+0000", 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0665999999999999e-05 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:00:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-16T13:00:56.214 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 1 2024-09-16T13:00:56.214 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:56.218 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:00:56.275 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:56.268+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:56.319 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:56.319 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:56.319 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:56.319 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:56.319 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:56.328 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:00:56.376 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:56.373+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:56.427 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:56.428 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:00:56.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:00:56.428 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2024-09-16T13:00:56.428 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:56.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:56.478 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:56.477+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:56.533 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T13:00:56.541 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T13:00:56.545 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T13:00:56.567 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:56.567 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:56.567 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:56.567 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:56.577 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:56.573+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:56.580 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:56.678 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:56.672+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:56.681 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:56.681 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:56.681 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:56.681 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:56.681 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:56.690 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:56.779 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:56.772+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:56.789 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:56.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093174005999998, 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.790 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:00:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:00:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:00:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:00:56.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:00:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:00:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:00:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:00:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093158979999998, 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:00:56.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:00:56.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:56.799 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093142958000001, 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:56.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:00:56.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093126861999998, 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093110971999998, 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.809 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:00:56.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:00:56.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093095302000002, 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.814 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:00:56.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:00:56.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:00:56.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.818 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093079555000003, 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093063766999997, 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:00:56.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:00:56.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:56.828 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093048445000001, 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:00:56.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:00:56.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.832 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093028519999997, 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:00:56.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.093012571999999, 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.841 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.092997410999999, 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:00:56.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.092981762999997, 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:00:56.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:00:56.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:56.850 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.092966666000002, 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:56.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:00:56.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:00:56.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.092951403999997, 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:56.859 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.092935658000002, 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:56.863 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.092920061000001, 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.092904756999999, 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.871 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.092889319000001, 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.092873605999998, 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:00:56.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:56.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:00:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:56.880 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:56.881 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:56.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:00:56.882 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:56.876+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:56.983 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:56.981+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:57.030 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:57.030 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:57.030 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:57.030 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:57.039 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:00:57.084 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:57.081+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:57.141 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:57.141 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:57.142 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:57.142 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:57.150 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:00:57.184 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:57.176+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:57.250 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254136 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 136192~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:00:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.396702799, 2024-09-16T13:00:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082717923999999998, 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254136 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849492+0000", 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-16T13:00:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849494+0000", 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0540000000000002e-06 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849500+0000", 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2899999999999999e-06 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849504+0000", 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8120000000000001e-06 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.931987+0000", 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082483231000000004 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932061+0000", 2024-09-16T13:00:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4181e-05 2024-09-16T13:00:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932209+0000", 2024-09-16T13:00:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014796399999999999 2024-09-16T13:00:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254137 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 139264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.396686524, 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082930443000000006, 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254137 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:00:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849510+0000", 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0530000000000001e-06 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849516+0000", 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8359999999999997e-06 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849520+0000", 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932239+0000", 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082719191999999997 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932307+0000", 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7824000000000007e-05 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932438+0000", 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000131354 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254138 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 146432~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.396670594, 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083100513000000001, 2024-09-16T13:00:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254138 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7300000000000001e-07 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849526+0000", 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9089999999999998e-06 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849532+0000", 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6420000000000004e-06 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849535+0000", 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7740000000000002e-06 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932467+0000", 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082931469999999993 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932535+0000", 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7815e-05 2024-09-16T13:00:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932624+0000", 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9430000000000003e-05 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254139 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 149504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:00:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.396655037, 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083267068, 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254139 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:00:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:00:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849540+0000", 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849542+0000", 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9379999999999999e-06 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849547+0000", 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849551+0000", 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5880000000000002e-06 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932652+0000", 2024-09-16T13:00:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083101176999999998 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932719+0000", 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.703e-05 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932806+0000", 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7340000000000001e-05 2024-09-16T13:00:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254140 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 156672~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.396639764, 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083436222000000004, 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254140 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849557+0000", 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8419999999999999e-06 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849564+0000", 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7079999999999997e-06 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849569+0000", 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.386e-06 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932833+0000", 2024-09-16T13:00:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083264099999999994 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932900+0000", 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7007000000000004e-05 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932991+0000", 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0815000000000004e-05 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254141 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.396621211999999, 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083599532000000004, 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254141 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:00:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849574+0000", 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1699999999999998e-07 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849576+0000", 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849582+0000", 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9769999999999997e-06 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849585+0000", 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933018+0000", 2024-09-16T13:00:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083432372000000005 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933086+0000", 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7862999999999997e-05 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933173+0000", 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7050000000000002e-05 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254142 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.396605062000001, 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083767603999999996, 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254142 2024-09-16T13:00:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849590+0000", 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.51e-07 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849592+0000", 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0360000000000001e-06 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849598+0000", 2024-09-16T13:00:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4139999999999999e-06 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849602+0000", 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.659e-06 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933199+0000", 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083597150999999995 2024-09-16T13:00:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933267+0000", 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7933000000000002e-05 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933357+0000", 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9859999999999997e-05 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254143 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.396588114, 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083932426000000004, 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254143 2024-09-16T13:00:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849607+0000", 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849609+0000", 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.165e-06 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849615+0000", 2024-09-16T13:00:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2029999999999997e-06 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849619+0000", 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933382+0000", 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083763447000000005 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933447+0000", 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4993000000000006e-05 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933539+0000", 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1434000000000006e-05 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254145 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.396571551999999, 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084130242999999993, 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254145 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851596+0000", 2024-09-16T13:00:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019732550000000001 2024-09-16T13:00:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851602+0000", 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5720000000000002e-06 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851610+0000", 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5529999999999993e-06 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851616+0000", 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6409999999999999e-06 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933580+0000", 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.081963854000000003 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933653+0000", 2024-09-16T13:00:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3901999999999998e-05 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933753+0000", 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9465999999999999e-05 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254149 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394555028999999, 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082661199000000005, 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254149 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851640+0000", 2024-09-16T13:00:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-16T13:00:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851642+0000", 2024-09-16T13:00:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-16T13:00:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851648+0000", 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9200000000000001e-06 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851652+0000", 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0439999999999997e-06 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934105+0000", 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082453495000000002 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934193+0000", 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7433000000000001e-05 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934300+0000", 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107513 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254151 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394533407999999, 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082910496, 2024-09-16T13:00:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254151 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:00:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1799999999999995e-07 2024-09-16T13:00:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851664+0000", 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4399999999999999e-06 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851670+0000", 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3509999999999998e-06 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851674+0000", 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.912e-06 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934385+0000", 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082710396000000005 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934471+0000", 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6040999999999998e-05 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934571+0000", 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100738 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254153 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394515954999999, 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083065172000000007, 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254153 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851679+0000", 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851681+0000", 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.137e-06 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851687+0000", 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.643e-06 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851690+0000", 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5580000000000001e-06 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934597+0000", 2024-09-16T13:00:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082906881000000002 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934663+0000", 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5828999999999996e-05 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934743+0000", 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0714e-05 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254155 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394500267, 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083254350000000005, 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254155 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:00:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851697+0000", 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.187e-06 2024-09-16T13:00:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851702+0000", 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7520000000000002e-06 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851706+0000", 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7979999999999999e-06 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934769+0000", 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083062834000000002 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934829+0000", 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0430999999999998e-05 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934948+0000", 2024-09-16T13:00:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118905 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254157 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 207872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394484342, 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083456394000000003, 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254157 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851712+0000", 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0150000000000002e-06 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851718+0000", 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5999999999999997e-06 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851722+0000", 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5810000000000001e-06 2024-09-16T13:00:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934992+0000", 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083270799000000006 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935066+0000", 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3851999999999997e-05 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935166+0000", 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100145 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254159 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 210944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394468996000001, 2024-09-16T13:00:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083625606000000005, 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254159 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851726+0000", 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-16T13:00:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851728+0000", 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851734+0000", 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7429999999999999e-06 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851738+0000", 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9620000000000004e-06 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935195+0000", 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083457518999999994 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935262+0000", 2024-09-16T13:00:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6465999999999997e-05 2024-09-16T13:00:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935351+0000", 2024-09-16T13:00:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9314000000000006e-05 2024-09-16T13:00:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254161 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 218112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394453005999999, 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083866861000000001, 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254161 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851742+0000", 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851744+0000", 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1550000000000001e-06 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851749+0000", 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5829999999999997e-06 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851753+0000", 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6990000000000001e-06 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935377+0000", 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083624025000000005 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935454+0000", 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7168000000000004e-05 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935608+0000", 2024-09-16T13:00:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153774 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254163 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 221184~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394437229999999, 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084055344000000004, 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254163 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851760+0000", 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2249999999999999e-06 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851765+0000", 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7799999999999997e-06 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851769+0000", 2024-09-16T13:00:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-06 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935654+0000", 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083884829999999994 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935723+0000", 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9163000000000002e-05 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935812+0000", 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9237e-05 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254164 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:00:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394421187000001, 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084726810999999999, 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254164 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851775+0000", 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.945e-06 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851781+0000", 2024-09-16T13:00:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851785+0000", 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6320000000000001e-06 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936053+0000", 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084268018 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936130+0000", 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7478999999999995e-05 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936500+0000", 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036965099999999997 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254165 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 228352~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394405663000001, 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084960919999999995, 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254165 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851791+0000", 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2819999999999999e-06 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851797+0000", 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7969999999999997e-06 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851801+0000", 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6679999999999999e-06 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936554+0000", 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084753203999999999 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936625+0000", 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1406e-05 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936750+0000", 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012416700000000001 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254166 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 6144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.394389852, 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.085115273000000005, 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254166 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856445+0000", 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046400929999999996 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856451+0000", 2024-09-16T13:00:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5119999999999997e-06 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856463+0000", 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2252999999999999e-05 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856470+0000", 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4160000000000001e-06 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936767+0000", 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080297258999999996 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936832+0000", 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4685999999999997e-05 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936920+0000", 2024-09-16T13:00:57.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8053999999999994e-05 2024-09-16T13:00:57.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.356 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.356 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.356 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:57.358 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:00:57.358 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:57.353+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:57.459 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:57.452+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:57.513 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:57.513 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:57.513 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:57.513 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:57.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:00:57.537 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T13:00:57.549 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T13:00:57.593 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:57.588+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:57.617 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:57.617 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:57.626 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:00:57.660 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:57.657+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:57.726 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:57.726 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:57.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:57.726 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:57.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.726 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260177 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.726 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:00:57.726 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.64526173, 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039279541000000001, 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260177 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:00:57.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075845+0000", 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075847+0000", 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075852+0000", 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.507e-06 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075856+0000", 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6389999999999998e-06 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.114965+0000", 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039109496000000001 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115034+0000", 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8378000000000002e-05 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115124+0000", 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0334999999999995e-05 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.730 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260178 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645246333999999, 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039530044, 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260178 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075866+0000", 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0909999999999998e-06 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075872+0000", 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.139e-06 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075876+0000", 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0419999999999996e-06 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115194+0000", 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039317193 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115259+0000", 2024-09-16T13:00:57.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5399000000000002e-05 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115390+0000", 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130803 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260179 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645225484000001, 2024-09-16T13:00:57.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039926426000000001, 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260179 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075883+0000", 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6959999999999999e-06 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075889+0000", 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9479999999999996e-06 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075893+0000", 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7840000000000001e-06 2024-09-16T13:00:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115419+0000", 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039526517999999997 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115577+0000", 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015783999999999999 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115807+0000", 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023010399999999999 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260180 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645209737, 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040245398000000002, 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260180 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:00:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075897+0000", 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075899+0000", 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8169999999999999e-06 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075904+0000", 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075908+0000", 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5889999999999999e-06 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115837+0000", 2024-09-16T13:00:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039929193000000002 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115978+0000", 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014115800000000001 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116142+0000", 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016352900000000001 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260181 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 180224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645194529999999, 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040482388000000001, 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260181 2024-09-16T13:00:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075914+0000", 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6759999999999999e-06 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075919+0000", 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2360000000000004e-06 2024-09-16T13:00:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075922+0000", 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116171+0000", 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040248702999999997 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116239+0000", 2024-09-16T13:00:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8391000000000003e-05 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116394+0000", 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000154575 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260182 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645179818999999, 2024-09-16T13:00:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042150168000000002, 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260182 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075927+0000", 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075929+0000", 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7859999999999999e-06 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075934+0000", 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7679999999999997e-06 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075938+0000", 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2590000000000001e-06 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116426+0000", 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040488077999999997 2024-09-16T13:00:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116493+0000", 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7681000000000005e-05 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118077+0000", 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015832260000000001 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260183 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645164831000001, 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040886082999999997, 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260183 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075942+0000", 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075943+0000", 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.778e-06 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075949+0000", 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-16T13:00:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075953+0000", 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116672+0000", 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040719563 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116743+0000", 2024-09-16T13:00:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0729000000000007e-05 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116827+0000", 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4364999999999996e-05 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260184 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645149668, 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041043524999999997, 2024-09-16T13:00:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260184 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075959+0000", 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6869999999999999e-06 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075964+0000", 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.242e-06 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075967+0000", 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116850+0000", 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040882977000000001 2024-09-16T13:00:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116916+0000", 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5413000000000005e-05 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117000+0000", 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4431999999999994e-05 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260185 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645134429000001, 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041177895999999999, 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260185 2024-09-16T13:00:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075974+0000", 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0049999999999999e-06 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075980+0000", 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7470000000000002e-06 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075984+0000", 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117022+0000", 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041038789999999999 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117090+0000", 2024-09-16T13:00:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7712000000000005e-05 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117150+0000", 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9542e-05 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260257 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645092575, 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041222406000000003, 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260257 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076017+0000", 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4990000000000001e-06 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076024+0000", 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076028+0000", 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9319999999999998e-06 2024-09-16T13:00:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117162+0000", 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04113472 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117199+0000", 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6507999999999997e-05 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117236+0000", 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7023000000000001e-05 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260258 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645074056, 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041343291999999997, 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260258 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:00:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:00:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076035+0000", 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.058e-06 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076041+0000", 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.998e-06 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076044+0000", 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3900000000000002e-06 2024-09-16T13:00:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117256+0000", 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041212555999999997 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117323+0000", 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6568000000000004e-05 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117375+0000", 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2361999999999998e-05 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260259 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645058638, 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041437587999999997, 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260259 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5499999999999999e-07 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076050+0000", 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076055+0000", 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5609999999999998e-06 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076059+0000", 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117396+0000", 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041336879999999999 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117454+0000", 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8443999999999998e-05 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117485+0000", 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1047999999999998e-05 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260260 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:00:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645043839, 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041496984000000001, 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260260 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:00:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076063+0000", 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076070+0000", 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7919999999999993e-06 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076076+0000", 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9359999999999996e-06 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076080+0000", 2024-09-16T13:00:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8240000000000001e-06 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117497+0000", 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041416896000000002 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117531+0000", 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4e-05 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117559+0000", 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8177000000000001e-05 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260261 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645021603, 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041555793000000001, 2024-09-16T13:00:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260261 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076087+0000", 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7850000000000001e-06 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076093+0000", 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6879999999999996e-06 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076096+0000", 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117570+0000", 2024-09-16T13:00:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041474329999999997 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117611+0000", 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0852999999999997e-05 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117640+0000", 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9213e-05 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260262 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.645006156999999, 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041643056999999997, 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260262 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076102+0000", 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8509999999999999e-06 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076108+0000", 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8560000000000003e-06 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076111+0000", 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0620000000000001e-06 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117651+0000", 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041539980999999997 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117703+0000", 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1975000000000003e-05 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117743+0000", 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9872e-05 2024-09-16T13:00:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260263 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.644991176, 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041724412000000002, 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260263 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076117+0000", 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.762e-06 2024-09-16T13:00:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076123+0000", 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076126+0000", 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.427e-06 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117757+0000", 2024-09-16T13:00:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041631230999999998 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117792+0000", 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4078999999999997e-05 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117839+0000", 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7861000000000001e-05 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260264 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.64497633, 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041783638999999997, 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260264 2024-09-16T13:00:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:00:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076132+0000", 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.751e-06 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076138+0000", 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6729999999999997e-06 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076141+0000", 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117851+0000", 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04171064 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117885+0000", 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3807999999999999e-05 2024-09-16T13:00:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117913+0000", 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8229999999999999e-05 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260265 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.644961727, 2024-09-16T13:00:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042095026000000001, 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260265 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076145+0000", 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076147+0000", 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076152+0000", 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5130000000000004e-06 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076156+0000", 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:00:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117924+0000", 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041768179000000002 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117980+0000", 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6125000000000002e-05 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118239+0000", 2024-09-16T13:00:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025934200000000003 2024-09-16T13:00:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260266 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.644946859000001, 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045273031999999998, 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260266 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076160+0000", 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:00:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076161+0000", 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.796e-06 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076167+0000", 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-06 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076171+0000", 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.579e-06 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.120966+0000", 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.044795297999999997 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121159+0000", 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192618 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121432+0000", 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000273663 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260267 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.644930481999999, 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046181535000000003, 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260267 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076178+0000", 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7969999999999999e-06 2024-09-16T13:00:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076184+0000", 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6570000000000002e-06 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076187+0000", 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.455e-06 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122253+0000", 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046065813999999997 2024-09-16T13:00:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122318+0000", 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5338000000000004e-05 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122357+0000", 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9088999999999998e-05 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:57.820 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:57.821 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:00:57.822 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:57.817+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:57.922 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:57.917+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:57.938 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:57.938 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:57.938 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:00:57.938 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:57.949 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:00:58.023 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:58.021+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:58.044 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:58.044 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:00:58.044 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:00:58.044 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:00:58.044 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:58.053 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:00:58.124 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:58.116+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:58.161 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:00:58.161 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:00:58.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:00:58.161 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:00:58.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:58.161 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:58.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.086795518000002, 2024-09-16T13:00:58.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:00:58.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:58.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:00:58.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.086743099000003, 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:00:58.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:00:58.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.086726251000002, 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:00:58.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:00:58.176 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.086710136999997, 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.177 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:00:58.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.086645206999997, 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:00:58.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.086629107999997, 2024-09-16T13:00:58.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:00:58.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:00:58.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:58.192 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.086555326999999, 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:00:58.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:00:58.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.086538126000001, 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:00:58.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.199 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:00:58.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:00:58.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.203 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.086521423000001, 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:00:58.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:00:58.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.550210168, 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:00:58.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:00:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:58.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:00:58.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:00:58.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:00:58.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:00:58.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:00:58.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:00:58.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.550193031999999, 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:00:58.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:58.220 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:58.221 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.550173449999999, 2024-09-16T13:00:58.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:00:58.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.221 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:00:58.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.222 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:58.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:00:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:00:58.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:58.226 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.550142591, 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:00:58.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:00:58.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.231 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.550074495, 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:00:58.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:00:58.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:00:58.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:00:58.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:00:58.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:00:58.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:00:58.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:00:58.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.542770640000001, 2024-09-16T13:00:58.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:00:58.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.238 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.238 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:00:58.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.238 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:00:58.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:00:58.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:00:58.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:00:58.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:00:58.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.542753050999998, 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:58.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:00:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:00:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:00:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.542738157999999, 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:00:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.542706541000001, 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:00:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:00:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:00:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:00:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.542688519999999, 2024-09-16T13:00:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:00:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:00:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.6286531480000002, 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2569454119999999, 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:00:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 279128 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:00:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527093+0000", 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4040000000000002e-06 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527104+0000", 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-05 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:00:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527133+0000", 2024-09-16T13:00:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8909999999999999e-05 2024-09-16T13:00:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:00:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527144+0000", 2024-09-16T13:00:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1497e-05 2024-09-16T13:00:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527223+0000", 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8577000000000003e-05 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527236+0000", 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3362e-05 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783387+0000", 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2561508619999999 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783699+0000", 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311687 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:00:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.784036+0000", 2024-09-16T13:00:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033771300000000001 2024-09-16T13:00:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:00:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:00:58.271 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:00:58.271 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:00:58.273 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:00:58.274 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:58.269+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:58.374 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:58.368+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:58.387 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:58.387 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:58.387 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:00:58.387 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:58.400 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:00:58.464 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:00:58.474 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:58.473+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:58.483 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:00:58.499 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:00:58.499 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:00:58.499 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:00:58.499 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:00:58.499 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:00:58.508 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:00:58.509 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:00:58.531 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:00:58.545 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T13:00:58.549 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T13:00:58.553 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T13:00:58.559 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:00:58.575 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:58.573+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:58.595 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:00:58.619 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:58.619 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:58.625 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:00:58.629 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:00:58.644 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:00:58.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:00:58.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:00:58.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:00:58.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:00:58.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:00:58.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:00:58.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:00:58.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:00:58.676 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:58.673+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:58.738 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:58.738 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:58.748 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:00:58.776 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:58.773+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:58.845 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:58.845 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:58.855 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:00:58.877 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:58.872+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:58.963 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:58.963 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:58.974 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:00:58.978 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:58.972+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:59.079 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:59.076+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:59.079 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:59.079 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:59.089 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:00:59.180 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:59.172+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:59.188 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:59.188 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:59.197 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:00:59.280 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:00:59.277+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:00:59.302 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:59.302 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:59.311 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:00:59.381 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:00:59.376+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:00:59.457 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:59.457 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:59.466 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:00:59.484 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:00:59.476+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:00:59.564 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:59.564 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:59.573 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:00:59.582 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:00:59.576+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:00:59.671 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:59.671 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:59.680 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:00:59.683 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:00:59.681+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:00:59.705 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T13:00:59.706 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T13:00:59.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T13:00:59.784 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:00:59.781+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:00:59.796 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:59.796 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:59.806 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:00:59.885 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:00:59.880+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:00:59.913 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:00:59.913 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:00:59.922 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:00:59.986 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:00:59.981+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:00.035 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:00.035 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:00.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:00.086 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:00.085+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:00.147 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:00.147 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:00.156 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:00.187 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:00.185+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:00.253 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:00.253 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:00.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:00.288 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:00.280+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:00.362 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:00.363 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:00.371 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:00.413 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:00.408+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:00.490 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:00.484+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:00.514 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:00.514 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:00.524 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:00.590 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:00.584+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:00.625 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:00.625 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:00.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:00.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:01:00.691 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:00.684+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:00.706 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T13:01:00.719 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T13:01:00.793 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:00.789+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:00.798 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:00.798 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:00.811 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:00.831 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:00.855 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:00.889 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:00.893 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:00.889+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:00.923 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:00.924 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:00.924 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:00.924 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:00.924 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:00.933 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:00.947 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:00.980 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:00.994 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:00.989+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:01.006 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:01:01.026 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:01:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:01.044 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:01.044 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:01.044 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:01.044 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:01.044 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:01.053 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:01.095 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:01.093+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:01.196 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:01.188+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:01.205 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:01.205 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:01.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:01.205 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:01.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.205 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.131503719999998, 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:01:01.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.131451300999998, 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:01:01.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:01:01.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.131434452999997, 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:01:01.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:01:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:01.219 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.131418339, 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:01.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:01:01.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:01.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:01:01.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:01:01.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:01.224 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.131353408999999, 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:01.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:01:01.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.131337309999999, 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:01.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:01:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.131263529000002, 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:01.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:01:01.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:01.238 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.131246328000003, 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:01.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:01:01.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.131229625000003, 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:01:01.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:01:01.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:01:01.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.594918370000002, 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.248 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:01:01.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.252 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.594901233999998, 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:01:01.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:01:01.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.257 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:01.258 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:01.258 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.594881651999998, 2024-09-16T13:01:01.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:01:01.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.258 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:01:01.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:01:01.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.594850792999999, 2024-09-16T13:01:01.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:01:01.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:01:01.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.594782696999999, 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.267 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:01:01.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:01:01.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.272 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:01.272 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:01.272 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.587478842000003, 2024-09-16T13:01:01.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:01:01.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:01:01.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:01:01.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:01:01.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:01:01.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:01:01.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.587461253000001, 2024-09-16T13:01:01.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:01:01.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.587446360000001, 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.587414742999997, 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.587396722000001, 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:01:01.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:01:01.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:01:01.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:01:01.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:01:01.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6733613500000004, 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2569454119999999, 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:01.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:01.297 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:01.297 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 279128 2024-09-16T13:01:01.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.297 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:01.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:01.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:01.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:01.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:01.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:01.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527093+0000", 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4040000000000002e-06 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527104+0000", 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-05 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527133+0000", 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8909999999999999e-05 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527144+0000", 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1497e-05 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527223+0000", 2024-09-16T13:01:01.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8577000000000003e-05 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527236+0000", 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3362e-05 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783387+0000", 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2561508619999999 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783699+0000", 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311687 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.784036+0000", 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033771300000000001 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:01.302 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:01.304 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:01.304 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:01.296+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:01.405 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:01.401+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:01.413 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:01.413 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:01.413 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:01.423 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:01.506 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:01.501+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:01.524 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:01.525 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:01.525 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:01.525 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:01.525 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:01.533 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:01.607 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:01.604+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:01.636 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:01.636 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:01.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:01.636 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:01.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.636 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240664 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 152576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.636 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:01.636 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.088600474, 2024-09-16T13:01:01.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047790817999999999, 2024-09-16T13:01:01.636 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240664 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1900000000000003e-07 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543064+0000", 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4449999999999999e-06 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543079+0000", 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5221e-05 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543084+0000", 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8590000000000004e-06 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590750+0000", 2024-09-16T13:01:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047665710999999999 2024-09-16T13:01:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590801+0000", 2024-09-16T13:01:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1827999999999999e-05 2024-09-16T13:01:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590852+0000", 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0235000000000003e-05 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240665 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.088571071000001, 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047896192999999997, 2024-09-16T13:01:01.641 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240665 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:01.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543091+0000", 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543094+0000", 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7149999999999998e-06 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543102+0000", 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6549999999999993e-06 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543108+0000", 2024-09-16T13:01:01.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4840000000000003e-06 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590885+0000", 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047777396 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590936+0000", 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0611e-05 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590987+0000", 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0785000000000003e-05 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240666 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 162816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.088548802999998, 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048010069000000002, 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240666 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8500000000000002e-07 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543116+0000", 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.531e-06 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543123+0000", 2024-09-16T13:01:01.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1069999999999999e-06 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543127+0000", 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4309999999999996e-06 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591020+0000", 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047892413000000002 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591071+0000", 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1650000000000002e-05 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591123+0000", 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1452000000000001e-05 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240667 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.088530021, 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048125951, 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240667 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543132+0000", 2024-09-16T13:01:01.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5099999999999999e-07 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543134+0000", 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1500000000000002e-06 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543141+0000", 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543146+0000", 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9200000000000003e-06 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591158+0000", 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048011246 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591207+0000", 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9295999999999997e-05 2024-09-16T13:01:01.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591257+0000", 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0426000000000001e-05 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240668 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 173056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.088510316000001, 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048243280999999999, 2024-09-16T13:01:01.655 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240668 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543152+0000", 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543154+0000", 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5380000000000001e-06 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543161+0000", 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7920000000000004e-06 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543172+0000", 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1327e-05 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591290+0000", 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048118181000000003 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591344+0000", 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4061000000000003e-05 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591394+0000", 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9866e-05 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240766 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.087305991000001, 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047716644000000003, 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240766 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544356+0000", 2024-09-16T13:01:01.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2500000000000005e-07 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544373+0000", 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6685999999999999e-05 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544381+0000", 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7190000000000001e-06 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544384+0000", 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591969+0000", 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047585483999999997 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592021+0000", 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1839000000000003e-05 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592072+0000", 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754000000000002e-05 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240767 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 201728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.087274116, 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047822479000000001, 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240767 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544388+0000", 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.51e-07 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544390+0000", 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.824e-06 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544393+0000", 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.681e-06 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544396+0000", 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4940000000000002e-06 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592105+0000", 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047709184000000002 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592158+0000", 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2772999999999998e-05 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592210+0000", 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2071999999999999e-05 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240768 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.668 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.087261314999999, 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047944647, 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240768 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0400000000000002e-07 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544403+0000", 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1430000000000001e-06 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544406+0000", 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7450000000000001e-06 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544409+0000", 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592241+0000", 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047832146999999998 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592291+0000", 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9289000000000002e-05 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592345+0000", 2024-09-16T13:01:01.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4067000000000003e-05 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240769 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.08724969, 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048066225999999997, 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240769 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1399999999999998e-07 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544413+0000", 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.42e-06 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544417+0000", 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.529e-06 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544420+0000", 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5620000000000002e-06 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592378+0000", 2024-09-16T13:01:01.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047958354000000002 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592427+0000", 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9013e-05 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592478+0000", 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1034e-05 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240770 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.087239427, 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048189432999999997, 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240770 2024-09-16T13:01:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3200000000000001e-07 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544424+0000", 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4449999999999999e-06 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544427+0000", 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5049999999999999e-06 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544430+0000", 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.61e-06 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592511+0000", 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048080912000000003 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592563+0000", 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2107000000000001e-05 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592611+0000", 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8522000000000001e-05 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.682 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240771 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.08720482, 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290128000000002, 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240771 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5700000000000002e-07 2024-09-16T13:01:01.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544459+0000", 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6840000000000001e-06 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544466+0000", 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999997e-06 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544469+0000", 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592645+0000", 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048175216999999999 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592696+0000", 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1292999999999998e-05 2024-09-16T13:01:01.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592747+0000", 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0751000000000002e-05 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240772 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.087181524999998, 2024-09-16T13:01:01.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048403333999999999, 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240772 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:01.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544482+0000", 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8390000000000001e-06 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544486+0000", 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9369999999999997e-06 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544489+0000", 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8459999999999999e-06 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592779+0000", 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290301000000001 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592831+0000", 2024-09-16T13:01:01.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2335e-05 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592883+0000", 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1786999999999997e-05 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240773 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.692 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.087170140000001, 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048527963, 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240773 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:01.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544492+0000", 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.84e-07 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544493+0000", 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6279999999999999e-06 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544498+0000", 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4689999999999999e-06 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544500+0000", 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592916+0000", 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048415672 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592965+0000", 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9120999999999999e-05 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593019+0000", 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3950999999999999e-05 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240774 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.087158485, 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048649546000000002, 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.697 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240774 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544505+0000", 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.61e-06 2024-09-16T13:01:01.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544508+0000", 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4929999999999998e-06 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544511+0000", 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4169999999999999e-06 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593051+0000", 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048539604 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593103+0000", 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2068999999999999e-05 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593152+0000", 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9870999999999997e-05 2024-09-16T13:01:01.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240775 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.087148231, 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048773328999999997, 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240775 2024-09-16T13:01:01.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545618+0000", 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011045810000000001 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545625+0000", 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6079999999999998e-06 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545644+0000", 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.823e-05 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545653+0000", 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3000000000000007e-06 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593186+0000", 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047533387000000003 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593235+0000", 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8767000000000003e-05 2024-09-16T13:01:01.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593286+0000", 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1456000000000003e-05 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240776 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.085996092999999, 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047754889000000002, 2024-09-16T13:01:01.706 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240776 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545666+0000", 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-06 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545671+0000", 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2270000000000004e-06 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545681+0000", 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0506000000000001e-05 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545687+0000", 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0970000000000002e-06 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593320+0000", 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047632999000000002 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593369+0000", 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8882999999999999e-05 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593420+0000", 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1127e-05 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240777 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.710 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.085967472, 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047878318000000003, 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240777 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545695+0000", 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545698+0000", 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0709999999999999e-06 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545707+0000", 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7629999999999995e-06 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545713+0000", 2024-09-16T13:01:01.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2839999999999997e-06 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593453+0000", 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047740281000000002 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593518+0000", 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5493000000000005e-05 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593572+0000", 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3739000000000002e-05 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240778 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.085943033, 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047999411999999998, 2024-09-16T13:01:01.715 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240778 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:01.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545719+0000", 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.13e-07 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545728+0000", 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1560000000000001e-06 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545901+0000", 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017311599999999999 2024-09-16T13:01:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545909+0000", 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.028e-06 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593606+0000", 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047696660000000002 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593666+0000", 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9611999999999998e-05 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593718+0000", 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2227e-05 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240951 4.8 4:10cc0d5f:::rbd_data.11b552946c0f.0000000000000120:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.077862763999999, 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048226894999999999, 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240951 2024-09-16T13:01:01.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:01.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2300000000000002e-07 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553801+0000", 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.686e-06 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553805+0000", 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3039999999999998e-06 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553809+0000", 2024-09-16T13:01:01.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.599994+0000", 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046185154999999999 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.600112+0000", 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118589 2024-09-16T13:01:01.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.602026+0000", 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019131370000000001 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:272626 4.12 4:4935e950:::rbd_data.11b59088eb24.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-16T13:01:01.725 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.173835142, 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022439990000001, 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 272626 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:01.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457828+0000", 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457835+0000", 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7900000000000002e-06 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457853+0000", 2024-09-16T13:01:01.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7671000000000001e-05 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457859+0000", 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2949999999999999e-06 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457913+0000", 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4112e-05 2024-09-16T13:01:01.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457922+0000", 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0850000000000003e-06 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.457879+0000", 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.99995641700000004 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458079+0000", 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020072300000000001 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458304+0000", 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000224601 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459393+0000", 2024-09-16T13:01:01.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010886170000000001 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459877+0000", 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048443500000000002 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460006+0000", 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012914 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:01.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:01.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460070+0000", 2024-09-16T13:01:01.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4074999999999999e-05 2024-09-16T13:01:01.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.733 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:01.733 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:01.733 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:01.734 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T13:01:01.735 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T13:01:01.736 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:01:01.736 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T13:01:01.737 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:01.728+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:01.835 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:01.833+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:01.837 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:01.837 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:01.838 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:01.838 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:01.847 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:01:01.936 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:01.933+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:01.996 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:01.996 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:01.996 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:01.997 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:01.997 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:02.005 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:01:02.051 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:02.049+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:5063 4.13 e45/31 4:ce724e7c:::rbd_data.11b53fd98b75.0000000000000000:head v 45'71, mlcod=41'69)", 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 94.739885013999995, 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056301611000000001, 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.107 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364194+0000", 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.364197+0000", 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8669999999999998e-06 2024-09-16T13:01:02.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393226+0000", 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029028953999999999 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393255+0000", 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9597000000000001e-05 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393266+0000", 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393342+0000", 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.393376+0000", 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4402e-05 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420434+0000", 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027057924000000001 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:27.420495+0000", 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1487000000000003e-05 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:18031 4.4 4:2649aabf:::rbd_data.11b51a3eb68b.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~5304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:01:02.111 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 66.943561781, 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94984657100000003, 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 18031 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160517+0000", 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160519+0000", 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3029999999999998e-06 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160536+0000", 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7181999999999999e-05 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160564+0000", 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971e-05 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160576+0000", 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160669+0000", 2024-09-16T13:01:02.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3282999999999994e-05 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:55.160680+0000", 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107188+0000", 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.94650794999999999 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.107644+0000", 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00045604499999999997 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.108257+0000", 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061324700000000003 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.109811+0000", 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015538749999999999 2024-09-16T13:01:02.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110175+0000", 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036419399999999998 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110285+0000", 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010938900000000001 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:59:56.110364+0000", 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8689000000000005e-05 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:64609 4.e 4:7394d9c8:::rbd_object_map.11b5c2ee3542:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e84)", 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.842352617000003, 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.020058463, 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.118 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 64609 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261726+0000", 2024-09-16T13:01:02.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261729+0000", 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4430000000000002e-06 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261741+0000", 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261780+0000", 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8722e-05 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261798+0000", 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8150000000000001e-05 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261927+0000", 2024-09-16T13:01:02.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000129243 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.261948+0000", 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1047e-05 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273712+0000", 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011763704999999999 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:19.273720+0000", 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4309999999999994e-06 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280059+0000", 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.006338744 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.280297+0000", 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000237667 2024-09-16T13:01:02.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281645+0000", 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001348555 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281702+0000", 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6342999999999999e-05 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:20.281785+0000", 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2782999999999997e-05 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254540 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 176128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816406+0000", 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287672393000001, 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053609236999999997, 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.125 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.126 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254540 2024-09-16T13:01:02.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.126 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816406+0000", 2024-09-16T13:01:02.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816406+0000", 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816407+0000", 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6000000000000004e-07 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816409+0000", 2024-09-16T13:01:02.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1349999999999999e-06 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816415+0000", 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9780000000000002e-06 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816424+0000", 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0990000000000005e-06 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.869803+0000", 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053378429999999998 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.869904+0000", 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101937 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870016+0000", 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111098 2024-09-16T13:01:02.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254541 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 183296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816431+0000", 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287647359000001, 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053814215999999998, 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.130 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254541 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816431+0000", 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816431+0000", 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816432+0000", 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816441+0000", 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8550000000000008e-06 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816457+0000", 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5509e-05 2024-09-16T13:01:02.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816463+0000", 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8800000000000002e-06 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870060+0000", 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053596442000000001 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870142+0000", 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2232999999999997e-05 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870246+0000", 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103509 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254542 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 186368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.134 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816469+0000", 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287609319000001, 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054024784999999999, 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254542 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816469+0000", 2024-09-16T13:01:02.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816469+0000", 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816470+0000", 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1999999999999998e-07 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816472+0000", 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.255e-06 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816479+0000", 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2539999999999998e-06 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816484+0000", 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.062e-06 2024-09-16T13:01:02.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870290+0000", 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053806132 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870379+0000", 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9128000000000005e-05 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870494+0000", 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115134 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254543 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 193536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287590940000001, 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054211546999999999, 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254543 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:01:02.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816488+0000", 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2e-07 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816490+0000", 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1220000000000002e-06 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816510+0000", 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9457999999999999e-05 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816522+0000", 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2585000000000001e-05 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870529+0000", 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054006981000000003 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870600+0000", 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0073999999999993e-05 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870699+0000", 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9807000000000002e-05 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254544 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 196608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816527+0000", 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287551758999999, 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054393364999999999, 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254544 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.144 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816527+0000", 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816527+0000", 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816528+0000", 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-07 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816530+0000", 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1619999999999998e-06 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816542+0000", 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2459e-05 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816555+0000", 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2532e-05 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870738+0000", 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054183505 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870818+0000", 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9425999999999995e-05 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870920+0000", 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010262700000000001 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254545 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 203776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816559+0000", 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287519256, 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054590611999999997, 2024-09-16T13:01:02.148 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254545 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816559+0000", 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816559+0000", 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816560+0000", 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9699999999999996e-07 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816562+0000", 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3549999999999999e-06 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816569+0000", 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3879999999999997e-06 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816573+0000", 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0720000000000001e-06 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.870958+0000", 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054385019 2024-09-16T13:01:02.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871029+0000", 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1291000000000006e-05 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871150+0000", 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012089 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254546 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 206848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816577+0000", 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287501762000002, 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054799678999999997, 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254546 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.153 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816577+0000", 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816577+0000", 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816578+0000", 2024-09-16T13:01:02.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6499999999999999e-07 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816580+0000", 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1270000000000001e-06 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816591+0000", 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1838000000000001e-05 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816595+0000", 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9349999999999996e-06 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871208+0000", 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054612978 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871275+0000", 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6774999999999997e-05 2024-09-16T13:01:02.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871377+0000", 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101461 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254547 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 214016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816600+0000", 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287478525000001, 2024-09-16T13:01:02.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054976553999999997, 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254547 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816600+0000", 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816600+0000", 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816601+0000", 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7000000000000005e-07 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816603+0000", 2024-09-16T13:01:02.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2500000000000001e-06 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816609+0000", 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9379999999999997e-06 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816615+0000", 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3860000000000004e-06 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871395+0000", 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054779732999999997 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871446+0000", 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0642000000000001e-05 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871577+0000", 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013103499999999999 2024-09-16T13:01:02.161 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254548 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 217088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816619+0000", 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287459260999999, 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055196303000000002, 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254548 2024-09-16T13:01:02.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816619+0000", 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816619+0000", 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816620+0000", 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2799999999999996e-07 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816622+0000", 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9999999999999999e-06 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816637+0000", 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4922e-05 2024-09-16T13:01:02.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816641+0000", 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1269999999999996e-06 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871623+0000", 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054981587999999998 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871688+0000", 2024-09-16T13:01:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5148999999999996e-05 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871816+0000", 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012798900000000001 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254549 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 224256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:01:02.166 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287403191999999, 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055369889999999998, 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254549 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:01:02.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816676+0000", 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7499999999999999e-07 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816679+0000", 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6120000000000001e-06 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816686+0000", 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816690+0000", 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0439999999999997e-06 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871868+0000", 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055177858000000003 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.871933+0000", 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4857000000000006e-05 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872045+0000", 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112858 2024-09-16T13:01:02.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254550 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 227328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287384955, 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055577646000000001, 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.171 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254550 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816694+0000", 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5599999999999995e-07 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816696+0000", 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9439999999999999e-06 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816702+0000", 2024-09-16T13:01:02.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7899999999999996e-06 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816707+0000", 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0520000000000001e-06 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872108+0000", 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055400479000000002 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872184+0000", 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6123000000000003e-05 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872271+0000", 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7701999999999995e-05 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.175 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254551 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 234496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816711+0000", 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287367665000001, 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055772654999999997, 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254551 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816711+0000", 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816711+0000", 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816712+0000", 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2e-07 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816714+0000", 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0770000000000002e-06 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816720+0000", 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0750000000000003e-06 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816725+0000", 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0069999999999997e-06 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872306+0000", 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055580967000000002 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872386+0000", 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9956999999999994e-05 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872484+0000", 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8052000000000002e-05 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254552 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 237568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816729+0000", 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287349890000002, 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055953530000000001, 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254552 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816729+0000", 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816729+0000", 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816730+0000", 2024-09-16T13:01:02.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0999999999999997e-07 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816732+0000", 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9769999999999999e-06 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816738+0000", 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0650000000000004e-06 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816743+0000", 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9969999999999998e-06 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872525+0000", 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055781981000000001 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872574+0000", 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9178000000000002e-05 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872682+0000", 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108522 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254553 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287332106000001, 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056166305999999999, 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.184 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254553 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816747+0000", 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2e-07 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816754+0000", 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9820000000000002e-06 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816760+0000", 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816764+0000", 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8940000000000003e-06 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872739+0000", 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055974278000000002 2024-09-16T13:01:02.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872798+0000", 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9564000000000001e-05 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872913+0000", 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114825 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254554 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 247808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816768+0000", 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287310480999999, 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056329235999999998, 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254554 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816768+0000", 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816768+0000", 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816769+0000", 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-07 2024-09-16T13:01:02.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816771+0000", 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.928e-06 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816777+0000", 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8819999999999998e-06 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816783+0000", 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4819999999999999e-06 2024-09-16T13:01:02.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.872935+0000", 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056152226 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873003+0000", 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7581000000000002e-05 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873097+0000", 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4482999999999996e-05 2024-09-16T13:01:02.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254555 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 254976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287278952000001, 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056501814999999997, 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254555 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.193 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816800+0000", 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8699999999999995e-07 2024-09-16T13:01:02.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816803+0000", 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.255e-06 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816809+0000", 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3119999999999999e-06 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816813+0000", 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.856e-06 2024-09-16T13:01:02.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873130+0000", 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056317045000000003 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873192+0000", 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1857000000000001e-05 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873302+0000", 2024-09-16T13:01:02.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010990299999999999 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:254556 4.18 4:1ea6552b:::rbd_data.11b55b44334a.0000000000000004:23 [read 258048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.287261938, 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056686382, 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:02.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 254556 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816817+0000", 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816819+0000", 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.046e-06 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816825+0000", 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8860000000000001e-06 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.816829+0000", 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873320+0000", 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056491373999999997 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873383+0000", 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2312999999999998e-05 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:43.873503+0000", 2024-09-16T13:01:02.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012061 2024-09-16T13:01:02.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.201 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.201 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.201 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:02.202 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:02.203 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:02.196+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:02.303 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:02.301+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:02.307 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:02.308 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:02.317 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:02.404 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:02.396+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:02.460 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:02.460 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:02.460 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:02.461 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:02.461 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:02.470 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:02.505 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:02.501+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:02.575 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:02.575 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:02.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:02.575 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:02.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.575 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:02.575 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:02.575 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 137.020312918, 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:02.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:01:02.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:01:02.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.19519587100001, 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.587 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.19514861499999, 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:01:02.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.194945679, 2024-09-16T13:01:02.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:01:02.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.19491441700001, 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.599 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.604 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.194878655, 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:01:02.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:01:02.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.19468402499999, 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.610 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:01:02.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.615 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.19463718, 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:01:02.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:01:02.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:01:02.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 135.65036100899999, 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:02.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:01:02.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:01:02.628 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.628 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.628 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.628 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:02.628 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:02.628 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 135.650307102, 2024-09-16T13:01:02.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:02.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:01:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:01:02.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 135.65026479700001, 2024-09-16T13:01:02.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:01:02.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:01:02.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 135.65010637200001, 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.640 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:01:02.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:01:02.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 135.65005689099999, 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:01:02.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 135.64992632100001, 2024-09-16T13:01:02.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:01:02.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:01:02.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:02.658 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 135.64974480800001, 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:02.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:01:02.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:01:02.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:01:02.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.83921714100001, 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:01:02.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:01:02.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.670 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.83906818700001, 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:01:02.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:01:02.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64507 4.1e e83/31 4:7f7c7834:::rbd_data.11b5c2ee3542.0000000000000008:head v 83'225, mlcod=81'223)", 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.7996549, 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021284401000000001, 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:02.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770380+0000", 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8999999999999998e-07 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777141+0000", 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067617670000000001 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777157+0000", 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777167+0000", 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777267+0000", 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100364 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777304+0000", 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7165000000000001e-05 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791601+0000", 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014297275999999999 2024-09-16T13:01:02.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791664+0000", 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2217000000000004e-05 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64509 4.15 e83/31 4:ad2e18f4:::rbd_data.11b5c2ee3542.0000000000000009:head v 83'183, mlcod=81'180)", 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.792896998000003, 2024-09-16T13:01:02.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014054395000000001, 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777138+0000", 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778534+0000", 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013956769999999999 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778555+0000", 2024-09-16T13:01:02.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1046000000000001e-05 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778564+0000", 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9290000000000004e-06 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778633+0000", 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9363000000000007e-05 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778666+0000", 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135000000000003e-05 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791145+0000", 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012479743 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791192+0000", 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6156000000000003e-05 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.685 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:217754 4.17 e92/31 4:ee4048fa:::rbd_data.11b58362ce83.0000000000000002:head v 92'697, mlcod=91'693)", 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.070726944, 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011416822, 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:02.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499309+0000", 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6929999999999999e-06 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499322+0000", 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499354+0000", 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135999999999999e-05 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499365+0000", 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111e-05 2024-09-16T13:01:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499504+0000", 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013877599999999999 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499565+0000", 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1939000000000004e-05 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510591+0000", 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011025874999999999 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510724+0000", 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013266299999999999 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:02.690 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:02.692 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:02.692 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:02.684+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:02.726 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T13:01:02.729 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T13:01:02.741 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T13:01:02.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:01:02.792 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:02.789+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:02.842 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:02.843 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:02.852 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:02.893 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:02.888+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:02.956 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:02.956 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:02.956 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:02.956 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:02.957 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:02.965 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:03.011 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:03.009+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:03.039 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:01:03.066 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:03.066 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:03.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:03.067 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:03.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.067 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-16T13:01:03.067 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:03.067 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 7.5261116040000005, 2024-09-16T13:01:03.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98766548899999995, 2024-09-16T13:01:03.067 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.067 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-16T13:01:03.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.068 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.068 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.068 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279128 2024-09-16T13:01:03.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.068 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:03.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:03.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534648+0000", 2024-09-16T13:01:03.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1459999999999999e-06 2024-09-16T13:01:03.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534655+0000", 2024-09-16T13:01:03.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5220000000000004e-06 2024-09-16T13:01:03.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534674+0000", 2024-09-16T13:01:03.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9717000000000001e-05 2024-09-16T13:01:03.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534683+0000", 2024-09-16T13:01:03.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:01:03.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534759+0000", 2024-09-16T13:01:03.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6074000000000004e-05 2024-09-16T13:01:03.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:03.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534770+0000", 2024-09-16T13:01:03.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0665999999999999e-05 2024-09-16T13:01:03.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522281+0000", 2024-09-16T13:01:03.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98751079600000002 2024-09-16T13:01:03.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522312+0000", 2024-09-16T13:01:03.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0876000000000001e-05 2024-09-16T13:01:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.9 109/108 [PushOp(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:head, version: 106'279, data_included: [0~475136], data_size: 475136, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:head@106'279, size: 475136, copy_subset: [0~475136], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:475136, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:56.636603+0000", 2024-09-16T13:01:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.4241547859999999, 2024-09-16T13:01:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026500030000000002, 2024-09-16T13:01:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.636603+0000", 2024-09-16T13:01:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.636603+0000", 2024-09-16T13:01:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.636604+0000", 2024-09-16T13:01:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3570000000000001e-06 2024-09-16T13:01:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.637425+0000", 2024-09-16T13:01:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082094999999999998 2024-09-16T13:01:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.637479+0000", 2024-09-16T13:01:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4249000000000002e-05 2024-09-16T13:01:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.637499+0000", 2024-09-16T13:01:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9103e-05 2024-09-16T13:01:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.637611+0000", 2024-09-16T13:01:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011230600000000001 2024-09-16T13:01:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.637621+0000", 2024-09-16T13:01:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8439999999999993e-06 2024-09-16T13:01:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.639253+0000", 2024-09-16T13:01:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001632194 2024-09-16T13:01:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.9 109/108 [PushOp(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29, version: 106'278, data_included: [0~688128,802816~10077,884736~295,1950271~32193], data_size: 730693, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29@106'278, size: 1982464, copy_subset: [0~1982464], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1982464, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.419092472, 2024-09-16T13:01:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067875690000000002, 2024-09-16T13:01:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641666+0000", 2024-09-16T13:01:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3060000000000001e-06 2024-09-16T13:01:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644856+0000", 2024-09-16T13:01:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00318995 2024-09-16T13:01:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644894+0000", 2024-09-16T13:01:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7382999999999998e-05 2024-09-16T13:01:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644913+0000", 2024-09-16T13:01:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9415999999999998e-05 2024-09-16T13:01:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645084+0000", 2024-09-16T13:01:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017036899999999999 2024-09-16T13:01:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645106+0000", 2024-09-16T13:01:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2887e-05 2024-09-16T13:01:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.648453+0000", 2024-09-16T13:01:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003346258 2024-09-16T13:01:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279212 4.9 4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:head [delete] snapc 29=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T13:01:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:57.882420+0000", 2024-09-16T13:01:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1783380619999999, 2024-09-16T13:01:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00335185, 2024-09-16T13:01:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279212 2024-09-16T13:01:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882420+0000", 2024-09-16T13:01:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882420+0000", 2024-09-16T13:01:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882423+0000", 2024-09-16T13:01:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4249999999999998e-06 2024-09-16T13:01:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882434+0000", 2024-09-16T13:01:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0702e-05 2024-09-16T13:01:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882458+0000", 2024-09-16T13:01:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4647999999999998e-05 2024-09-16T13:01:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882473+0000", 2024-09-16T13:01:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4693e-05 2024-09-16T13:01:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882601+0000", 2024-09-16T13:01:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012783299999999999 2024-09-16T13:01:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882779+0000", 2024-09-16T13:01:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000178464 2024-09-16T13:01:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.883026+0000", 2024-09-16T13:01:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024698400000000001 2024-09-16T13:01:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.884874+0000", 2024-09-16T13:01:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018472880000000001 2024-09-16T13:01:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.885447+0000", 2024-09-16T13:01:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057355199999999996 2024-09-16T13:01:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.885568+0000", 2024-09-16T13:01:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120567 2024-09-16T13:01:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.885771+0000", 2024-09-16T13:01:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020369400000000001 2024-09-16T13:01:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.18 110/109 [PushOp(4:1df486ae:::rbd_data.11b5de89dd5b.0000000000000000:21, version: 93'279, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:1df486ae:::rbd_data.11b5de89dd5b.0000000000000000:21@93'279, size: 14680064, copy_subset: [0~14680064], clone_subset: {}, snapset: 21=[]:{21=[21]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:58.006839+0000", 2024-09-16T13:01:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0539186950000001, 2024-09-16T13:01:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025323519999999999, 2024-09-16T13:01:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.006839+0000", 2024-09-16T13:01:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.006839+0000", 2024-09-16T13:01:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.006840+0000", 2024-09-16T13:01:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:01:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.007721+0000", 2024-09-16T13:01:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088152100000000002 2024-09-16T13:01:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.007750+0000", 2024-09-16T13:01:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8852000000000001e-05 2024-09-16T13:01:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.007764+0000", 2024-09-16T13:01:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3434999999999999e-05 2024-09-16T13:01:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.007881+0000", 2024-09-16T13:01:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116971 2024-09-16T13:01:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.007893+0000", 2024-09-16T13:01:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.203e-05 2024-09-16T13:01:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.009371+0000", 2024-09-16T13:01:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001478579 2024-09-16T13:01:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.18 110/109 [PushOp(4:1df486ae:::rbd_data.11b5de89dd5b.0000000000000000:21, version: 93'279, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:1df486ae:::rbd_data.11b5de89dd5b.0000000000000000:21@93'279, size: 14680064, copy_subset: [0~14680064], clone_subset: {}, snapset: 21=[]:{21=[21]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:14680064, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T13:01:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:58.010777+0000", 2024-09-16T13:01:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0499807790000002, 2024-09-16T13:01:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024390649999999998, 2024-09-16T13:01:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.010777+0000", 2024-09-16T13:01:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.010777+0000", 2024-09-16T13:01:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.010778+0000", 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.116e-06 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.011604+0000", 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082551599999999999 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.011634+0000", 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0685000000000002e-05 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.011647+0000", 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.279e-05 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.011759+0000", 2024-09-16T13:01:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011243699999999999 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.011768+0000", 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4519999999999997e-06 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.013216+0000", 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001448069 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.d 110/109 [PushOp(4:b0c4b67f:::rbd_data.11b5403f243a.0000000000000089:1f, version: 88'215, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:b0c4b67f:::rbd_data.11b5403f243a.0000000000000089:1f@88'215, size: 524288, copy_subset: [0~524288], clone_subset: {}, snapset: 1f=[]:{1f=[1f]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:524288, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:58.147166+0000", 2024-09-16T13:01:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9135916440000003, 2024-09-16T13:01:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002462929, 2024-09-16T13:01:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.147166+0000", 2024-09-16T13:01:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.147166+0000", 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.147167+0000", 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.235e-06 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.147982+0000", 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081512299999999995 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.148018+0000", 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5306999999999999e-05 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.148032+0000", 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4674e-05 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.148147+0000", 2024-09-16T13:01:03.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115037 2024-09-16T13:01:03.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.148156+0000", 2024-09-16T13:01:03.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9949999999999994e-06 2024-09-16T13:01:03.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:58.149629+0000", 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014725579999999999 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279642 4.d 4:b54cfc8a:::rbd_header.11b5a3d495b5:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:01.711059+0000", 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3496989529999999, 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035981020000000002, 2024-09-16T13:01:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279642 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711059+0000", 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711059+0000", 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711062+0000", 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0139999999999999e-06 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711078+0000", 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5988e-05 2024-09-16T13:01:03.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711113+0000", 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5355999999999998e-05 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711130+0000", 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6866000000000001e-05 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711242+0000", 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112358 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711405+0000", 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016311 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.712033+0000", 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00062738500000000005 2024-09-16T13:01:03.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.713048+0000", 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001015232 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.714543+0000", 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014950180000000001 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.714594+0000", 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1248000000000001e-05 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.714657+0000", 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2527000000000006e-05 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279663 4.d 4:b54cfc8a:::rbd_header.11b5a3d495b5:head [watch watch cookie 140717154789840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:01.742441+0000", 2024-09-16T13:01:03.118 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3183168940000001, 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025437929999999999, 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279663 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.742441+0000", 2024-09-16T13:01:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.742441+0000", 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.742442+0000", 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.742446+0000", 2024-09-16T13:01:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3590000000000001e-06 2024-09-16T13:01:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.742459+0000", 2024-09-16T13:01:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2891000000000001e-05 2024-09-16T13:01:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.742466+0000", 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2099999999999996e-06 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.742507+0000", 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0890000000000003e-05 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.742570+0000", 2024-09-16T13:01:03.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2343999999999998e-05 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.742700+0000", 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013020299999999999 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.743528+0000", 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000828471 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.744773+0000", 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001245173 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.744868+0000", 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4401000000000006e-05 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.744985+0000", 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116737 2024-09-16T13:01:03.124 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279689 4.d 4:b54cfc8a:::rbd_header.11b5a3d495b5:head [watch watch cookie 140717156667040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:01.777245+0000", 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.283512797, 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025105739999999998, 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.125 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279689 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.777245+0000", 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.777245+0000", 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.777247+0000", 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7600000000000001e-06 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.777255+0000", 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0369999999999995e-06 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.777284+0000", 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9227999999999999e-05 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.777299+0000", 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4797000000000001e-05 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.777400+0000", 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101334 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.777552+0000", 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015175099999999999 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.777770+0000", 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000218443 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.778976+0000", 2024-09-16T13:01:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012059830000000001 2024-09-16T13:01:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.779537+0000", 2024-09-16T13:01:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056065399999999995 2024-09-16T13:01:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.779638+0000", 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010077 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.779755+0000", 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117817 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279692 4.d 4:b54cfc8a:::rbd_header.11b5a3d495b5:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T13:01:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:01.781171+0000", 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2795863679999999, 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002530727, 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279692 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.781171+0000", 2024-09-16T13:01:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.781171+0000", 2024-09-16T13:01:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.781173+0000", 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4339999999999999e-06 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.781180+0000", 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3459999999999996e-06 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.781199+0000", 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8953999999999998e-05 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.781209+0000", 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.594e-06 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.781294+0000", 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5000999999999995e-05 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.781442+0000", 2024-09-16T13:01:03.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014868499999999999 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.781768+0000", 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032544900000000001 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.782805+0000", 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001036868 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.783542+0000", 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073735399999999996 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.783630+0000", 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8115000000000006e-05 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.783702+0000", 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1927000000000004e-05 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.137 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279698 4.d 4:b54cfc8a:::rbd_header.11b5a3d495b5:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:01.790843+0000", 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2699143479999999, 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026359270000000001, 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279698 2024-09-16T13:01:03.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790843+0000", 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790843+0000", 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790845+0000", 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.81e-06 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790855+0000", 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8830000000000001e-06 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790883+0000", 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7552e-05 2024-09-16T13:01:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790896+0000", 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3540000000000001e-05 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790988+0000", 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1791000000000003e-05 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.791141+0000", 2024-09-16T13:01:03.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000153423 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.791473+0000", 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000331336 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.792534+0000", 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010613280000000001 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793355+0000", 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082062699999999999 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793411+0000", 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6697000000000003e-05 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793479+0000", 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7940000000000003e-05 2024-09-16T13:01:03.143 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279700 4.d 4:b54cfc8a:::rbd_header.11b5a3d495b5:head [watch unwatch cookie 140717156667040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:01.793751+0000", 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2670070339999999, 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025856249999999998, 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279700 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793751+0000", 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793751+0000", 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793752+0000", 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.296e-06 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793757+0000", 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793790+0000", 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2265999999999999e-05 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793797+0000", 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5129999999999999e-06 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794104+0000", 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030651300000000001 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794315+0000", 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021124499999999999 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794580+0000", 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026477400000000001 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.795644+0000", 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010640809999999999 2024-09-16T13:01:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.796164+0000", 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00051998500000000004 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.796219+0000", 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4975000000000001e-05 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.796336+0000", 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011743699999999999 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279701 4.d 4:b54cfc8a:::rbd_header.11b5a3d495b5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:01.794215+0000", 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2665424490000001, 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024876469999999999, 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279701 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794215+0000", 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794215+0000", 2024-09-16T13:01:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794217+0000", 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2299999999999998e-06 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794228+0000", 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893000000000001e-05 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794264+0000", 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5642999999999997e-05 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794279+0000", 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5196e-05 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.795271+0000", 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000991598 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.795337+0000", 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5784000000000006e-05 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.796417+0000", 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010806769999999999 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.796552+0000", 2024-09-16T13:01:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135167 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.796703+0000", 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150459 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279793 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.744160+0000", 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.31659726900000001, 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004380274, 2024-09-16T13:01:03.155 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279793 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744160+0000", 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744160+0000", 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744163+0000", 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.067e-06 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744176+0000", 2024-09-16T13:01:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2184999999999999e-05 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744207+0000", 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1754000000000001e-05 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744219+0000", 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1457999999999999e-05 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744341+0000", 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122101 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744351+0000", 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0222e-05 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.748076+0000", 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037249679999999999 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.748277+0000", 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020039299999999999 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.748541+0000", 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000264126 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279845 4.1b 4:d9fcaac3:::rbd_id.image_client.0-clone33:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.822177+0000", 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.23858043300000001, 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034811210000000002, 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279845 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822177+0000", 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822177+0000", 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822181+0000", 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5429999999999998e-06 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822196+0000", 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5676999999999999e-05 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822241+0000", 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4082000000000002e-05 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822259+0000", 2024-09-16T13:01:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8216e-05 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822414+0000", 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015485 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822684+0000", 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027027000000000003 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822951+0000", 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026715000000000002 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.824738+0000", 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017868440000000001 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.825411+0000", 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067339799999999997 2024-09-16T13:01:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.825582+0000", 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017053899999999999 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.825658+0000", 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6551999999999995e-05 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279882 4.c 4:3656acfa:::rbd_data.11b53ee6e6bb.0000000000000009:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.19211530800000001, 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004921407, 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279882 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868645+0000", 2024-09-16T13:01:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5550000000000001e-06 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868656+0000", 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868691+0000", 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4459999999999999e-05 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868703+0000", 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2556e-05 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868818+0000", 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011502000000000001 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.869209+0000", 2024-09-16T13:01:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039085699999999999 2024-09-16T13:01:03.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:03.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.869563+0000", 2024-09-16T13:01:03.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035391699999999998 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.871027+0000", 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00146393 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873450+0000", 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002422914 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873527+0000", 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7286000000000005e-05 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873564+0000", 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6553000000000001e-05 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279890 4.e 4:726a396f:::rbd_data.11b53ee6e6bb.0000000000000017:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.873597+0000", 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.18716090099999999, 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028654750000000001, 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279890 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873597+0000", 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873597+0000", 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873599+0000", 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3140000000000002e-06 2024-09-16T13:01:03.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873611+0000", 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1491000000000001e-05 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873633+0000", 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2152000000000001e-05 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873644+0000", 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1664999999999999e-05 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873734+0000", 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9480000000000004e-05 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873966+0000", 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000231695 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.874176+0000", 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021012299999999999 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.875582+0000", 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001406205 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876374+0000", 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00079198300000000001 2024-09-16T13:01:03.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876423+0000", 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8875999999999998e-05 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876462+0000", 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9490999999999998e-05 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279895 4.1 4:848811f6:::rbd_data.11b53ee6e6bb.000000000000003c:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.876321+0000", 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.18443684199999999, 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003317801, 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279895 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876321+0000", 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876321+0000", 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876322+0000", 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5719999999999999e-06 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876330+0000", 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8720000000000002e-06 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876351+0000", 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0820999999999999e-05 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876362+0000", 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0558e-05 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876464+0000", 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102446 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876729+0000", 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533900000000002 2024-09-16T13:01:03.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.877078+0000", 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034812899999999997 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.879424+0000", 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023464250000000001 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.879500+0000", 2024-09-16T13:01:03.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5981000000000003e-05 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.879579+0000", 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8870999999999998e-05 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.879639+0000", 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9787000000000002e-05 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.185 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279967 4.1 4:8218f49a:::rbd_object_map.11b552946c0f.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.951941+0000", 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.108817106, 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025624770000000001, 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279967 2024-09-16T13:01:03.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951941+0000", 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951941+0000", 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:03.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951942+0000", 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.403e-06 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951948+0000", 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2879999999999998e-06 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951969+0000", 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0350999999999999e-05 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951981+0000", 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2557e-05 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.952140+0000", 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015917500000000001 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.952286+0000", 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014593000000000001 2024-09-16T13:01:03.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.952469+0000", 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018234499999999999 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.953732+0000", 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012638930000000001 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.954366+0000", 2024-09-16T13:01:03.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063367499999999999 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.954445+0000", 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8572000000000006e-05 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.954503+0000", 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8288000000000002e-05 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:03.192 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:03.192 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:03.195 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:01:03.197 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:01:03.197 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-16T13:01:02.714165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1221106116,"num_objects":2363,"num_object_clones":409,"num_object_copies":4726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":1252,"num_read":327224,"num_read_kb":642404,"num_write":19175,"num_write_kb":309057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":771,"num_bytes_recovered":24015535,"num_keys_recovered":147,"num_objects_omap":39,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16375,"ondisk_log_size":16375,"up":87,"acting":87,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2472152,"kb_used_data":49920,"kb_used_omap":47,"kb_used_meta":2422032,"kb_avail":752502568,"statfs":{"total":773094113280,"available":770562629632,"internally_reserved":0,"allocated":51118080,"data_stored":330189298,"data_compressed":8522211,"data_compressed_allocated":22171648,"data_compressed_original":310385094,"omap_allocated":48222,"internal_metadata":2480161698},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-47105986,"num_objects":-104,"num_object_clones":-45,"num_object_copies":-208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-104,"num_whiteouts":-37,"num_read":611,"num_read_kb":883,"num_write":127,"num_write_kb":3109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":16103119,"num_keys_recovered":36,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028122"},"pg_stats":[{"pgid":"4.1f","version":"110'282","reported_seq":3987,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.697585+0000","last_change":"2024-09-16T13:00:59.698023+0000","last_active":"2024-09-16T13:00:59.697585+0000","last_peered":"2024-09-16T13:00:59.697585+0000","last_clean":"2024-09-16T13:00:59.697585+0000","last_became_active":"2024-09-16T12:59:11.766798+0000","last_became_peered":"2024-09-16T12:59:11.766798+0000","last_unstale":"2024-09-16T13:00:59.697585+0000","last_undegraded":"2024-09-16T13:00:59.697585+0000","last_fullsized":"2024-09-16T13:00:59.697585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:22.588888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077473400000000001,"stat_sum":{"num_bytes":20966621,"num_objects":62,"num_object_clones":14,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":30,"num_read":3915,"num_read_kb":19318,"num_write":302,"num_write_kb":5430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1e","version":"113'432","reported_seq":16853,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.701881+0000","last_change":"2024-09-16T13:00:59.702141+0000","last_active":"2024-09-16T13:00:59.701881+0000","last_peered":"2024-09-16T13:00:59.701881+0000","last_clean":"2024-09-16T13:00:59.701881+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:00:59.701881+0000","last_undegraded":"2024-09-16T13:00:59.701881+0000","last_fullsized":"2024-09-16T13:00:59.701881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0088928189999999997,"stat_sum":{"num_bytes":47164207,"num_objects":89,"num_object_clones":18,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":46,"num_read":17148,"num_read_kb":19148,"num_write":434,"num_write_kb":9454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1d","version":"113'391","reported_seq":4743,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.719982+0000","last_change":"2024-09-16T13:00:59.720537+0000","last_active":"2024-09-16T13:00:59.719982+0000","last_peered":"2024-09-16T13:00:59.719982+0000","last_clean":"2024-09-16T13:00:59.719982+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:00:59.719982+0000","last_undegraded":"2024-09-16T13:00:59.719982+0000","last_fullsized":"2024-09-16T13:00:59.719982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.019188515999999999,"stat_sum":{"num_bytes":26736354,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":5128,"num_read_kb":15486,"num_write":330,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1c","version":"113'1047","reported_seq":7454,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.723859+0000","last_change":"2024-09-16T13:00:59.723859+0000","last_active":"2024-09-16T13:00:59.723859+0000","last_peered":"2024-09-16T13:00:59.723859+0000","last_clean":"2024-09-16T13:00:59.723859+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:00:59.723859+0000","last_undegraded":"2024-09-16T13:00:59.723859+0000","last_fullsized":"2024-09-16T13:00:59.723859+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.0076727519999999997,"stat_sum":{"num_bytes":23416045,"num_objects":81,"num_object_clones":10,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":45,"num_read":9824,"num_read_kb":15446,"num_write":1401,"num_write_kb":5620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1b","version":"111'385","reported_seq":21486,"reported_epoch":112,"state":"peering","last_fresh":"2024-09-16T13:00:58.693920+0000","last_change":"2024-09-16T13:00:58.690464+0000","last_active":"2024-09-16T13:00:58.689474+0000","last_peered":"2024-09-16T13:00:58.610355+0000","last_clean":"2024-09-16T13:00:18.903627+0000","last_became_active":"2024-09-16T13:00:57.780913+0000","last_became_peered":"2024-09-16T13:00:57.780913+0000","last_unstale":"2024-09-16T13:00:58.693920+0000","last_undegraded":"2024-09-16T13:00:58.693920+0000","last_fullsized":"2024-09-16T13:00:58.693920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:55.299561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002573915,"stat_sum":{"num_bytes":51579248,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":42,"num_read":22373,"num_read_kb":44612,"num_write":369,"num_write_kb":7677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"106'597","reported_seq":6563,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:54.999280+0000","last_change":"2024-09-16T13:00:20.273695+0000","last_active":"2024-09-16T13:00:54.999280+0000","last_peered":"2024-09-16T13:00:54.999280+0000","last_clean":"2024-09-16T13:00:17.071817+0000","last_became_active":"2024-09-16T13:00:20.273695+0000","last_became_peered":"2024-09-16T13:00:20.273695+0000","last_unstale":"2024-09-16T13:00:54.999280+0000","last_undegraded":"2024-09-16T13:00:20.272075+0000","last_fullsized":"2024-09-16T13:00:20.271923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:03.132002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002565964,"stat_sum":{"num_bytes":51437972,"num_objects":90,"num_object_clones":19,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":90,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":42,"num_read":7614,"num_read_kb":16910,"num_write":835,"num_write_kb":9749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":90}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"113'349","reported_seq":5222,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.715344+0000","last_change":"2024-09-16T13:00:59.716099+0000","last_active":"2024-09-16T13:00:59.715344+0000","last_peered":"2024-09-16T13:00:59.715344+0000","last_clean":"2024-09-16T13:00:59.715344+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:00:59.715344+0000","last_undegraded":"2024-09-16T13:00:59.715344+0000","last_fullsized":"2024-09-16T13:00:59.715344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01521717,"stat_sum":{"num_bytes":20728887,"num_objects":71,"num_object_clones":16,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":37,"num_read":4887,"num_read_kb":18652,"num_write":320,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.18","version":"112'361","reported_seq":6111,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.785057+0000","last_change":"2024-09-16T13:00:58.697117+0000","last_active":"2024-09-16T13:00:58.785057+0000","last_peered":"2024-09-16T13:00:58.785057+0000","last_clean":"2024-09-16T13:00:58.785057+0000","last_became_active":"2024-09-16T13:00:58.691043+0000","last_became_peered":"2024-09-16T13:00:58.691043+0000","last_unstale":"2024-09-16T13:00:58.785057+0000","last_undegraded":"2024-09-16T13:00:58.785057+0000","last_fullsized":"2024-09-16T13:00:58.785057+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:32.993895+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0038786659999999998,"stat_sum":{"num_bytes":49330762,"num_objects":78,"num_object_clones":15,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":36,"num_read":6289,"num_read_kb":20336,"num_write":427,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.17","version":"108'780","reported_seq":7239,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.703198+0000","last_change":"2024-09-16T13:00:59.708220+0000","last_active":"2024-09-16T13:00:59.703198+0000","last_peered":"2024-09-16T13:00:59.703198+0000","last_clean":"2024-09-16T13:00:59.703198+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:00:59.703198+0000","last_undegraded":"2024-09-16T13:00:59.703198+0000","last_fullsized":"2024-09-16T13:00:59.703198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007678603,"stat_sum":{"num_bytes":64984009,"num_objects":85,"num_object_clones":12,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":42,"num_read":10635,"num_read_kb":28964,"num_write":967,"num_write_kb":16664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.16","version":"113'405","reported_seq":2744,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.709331+0000","last_change":"2024-09-16T13:00:59.709660+0000","last_active":"2024-09-16T13:00:59.709331+0000","last_peered":"2024-09-16T13:00:59.709331+0000","last_clean":"2024-09-16T13:00:59.709331+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:00:59.709331+0000","last_undegraded":"2024-09-16T13:00:59.709331+0000","last_fullsized":"2024-09-16T13:00:59.709331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016073513000000001,"stat_sum":{"num_bytes":26333234,"num_objects":73,"num_object_clones":10,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":44,"num_read":4424,"num_read_kb":11797,"num_write":357,"num_write_kb":2394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.15","version":"113'434","reported_seq":15841,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.701013+0000","last_change":"2024-09-16T13:00:59.702236+0000","last_active":"2024-09-16T13:00:59.701013+0000","last_peered":"2024-09-16T13:00:59.701013+0000","last_clean":"2024-09-16T13:00:59.701013+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:00:59.701013+0000","last_undegraded":"2024-09-16T13:00:59.701013+0000","last_fullsized":"2024-09-16T13:00:59.701013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0086595639999999998,"stat_sum":{"num_bytes":41039232,"num_objects":69,"num_object_clones":10,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":40,"num_read":16194,"num_read_kb":17487,"num_write":499,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.14","version":"113'477","reported_seq":22835,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.712692+0000","last_change":"2024-09-16T13:00:59.713166+0000","last_active":"2024-09-16T13:00:59.712692+0000","last_peered":"2024-09-16T13:00:59.712692+0000","last_clean":"2024-09-16T13:00:59.712692+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:00:59.712692+0000","last_undegraded":"2024-09-16T13:00:59.712692+0000","last_fullsized":"2024-09-16T13:00:59.712692+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012577595,"stat_sum":{"num_bytes":68334983,"num_objects":80,"num_object_clones":18,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":39,"num_read":23272,"num_read_kb":34870,"num_write":526,"num_write_kb":16253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.13","version":"106'399","reported_seq":5557,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:54.521735+0000","last_change":"2024-09-16T13:00:20.278920+0000","last_active":"2024-09-16T13:00:54.521735+0000","last_peered":"2024-09-16T13:00:54.521735+0000","last_clean":"2024-09-16T13:00:16.244289+0000","last_became_active":"2024-09-16T13:00:20.278920+0000","last_became_peered":"2024-09-16T13:00:20.278920+0000","last_unstale":"2024-09-16T13:00:54.521735+0000","last_undegraded":"2024-09-16T13:00:20.278208+0000","last_fullsized":"2024-09-16T13:00:20.277996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:09.910764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024518000000000001,"stat_sum":{"num_bytes":55427842,"num_objects":71,"num_object_clones":13,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":71,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":33,"num_read":6200,"num_read_kb":27942,"num_write":410,"num_write_kb":42965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":71}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"107'329","reported_seq":2876,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.693420+0000","last_change":"2024-09-16T13:00:59.694422+0000","last_active":"2024-09-16T13:00:59.693420+0000","last_peered":"2024-09-16T13:00:59.693420+0000","last_clean":"2024-09-16T13:00:59.693420+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:00:59.693420+0000","last_undegraded":"2024-09-16T13:00:59.693420+0000","last_fullsized":"2024-09-16T13:00:59.693420+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013516940000000001,"stat_sum":{"num_bytes":39335036,"num_objects":74,"num_object_clones":9,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":52,"num_read":2795,"num_read_kb":14285,"num_write":359,"num_write_kb":3309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.11","version":"113'307","reported_seq":18494,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.711534+0000","last_change":"2024-09-16T13:00:59.712072+0000","last_active":"2024-09-16T13:00:59.711534+0000","last_peered":"2024-09-16T13:00:59.711534+0000","last_clean":"2024-09-16T13:00:59.711534+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:00:59.711534+0000","last_undegraded":"2024-09-16T13:00:59.711534+0000","last_fullsized":"2024-09-16T13:00:59.711534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.012068730999999999,"stat_sum":{"num_bytes":60439342,"num_objects":68,"num_object_clones":9,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":42,"num_read":18391,"num_read_kb":29155,"num_write":369,"num_write_kb":9723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.10","version":"111'289","reported_seq":5201,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.701580+0000","last_change":"2024-09-16T13:00:59.702319+0000","last_active":"2024-09-16T13:00:59.701580+0000","last_peered":"2024-09-16T13:00:59.701580+0000","last_clean":"2024-09-16T13:00:59.701580+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:00:59.701580+0000","last_undegraded":"2024-09-16T13:00:59.701580+0000","last_fullsized":"2024-09-16T13:00:59.701580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023312129999999999,"stat_sum":{"num_bytes":16079758,"num_objects":73,"num_object_clones":10,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":48,"num_read":5031,"num_read_kb":16220,"num_write":291,"num_write_kb":2924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.690056+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T13:00:58.690056+0000","last_peered":"2024-09-16T13:00:58.690056+0000","last_clean":"2024-09-16T13:00:58.690056+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T13:00:58.690056+0000","last_undegraded":"2024-09-16T13:00:58.690056+0000","last_fullsized":"2024-09-16T13:00:58.690056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"112'357","reported_seq":3247,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.698899+0000","last_change":"2024-09-16T13:00:58.699591+0000","last_active":"2024-09-16T13:00:58.698899+0000","last_peered":"2024-09-16T13:00:58.698899+0000","last_clean":"2024-09-16T13:00:58.698899+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:00:58.698899+0000","last_undegraded":"2024-09-16T13:00:58.698899+0000","last_fullsized":"2024-09-16T13:00:58.698899+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062261210000000003,"stat_sum":{"num_bytes":30423788,"num_objects":59,"num_object_clones":6,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":34,"num_read":3947,"num_read_kb":3256,"num_write":300,"num_write_kb":3627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622085+0000","last_change":"2024-09-16T12:59:40.993864+0000","last_active":"2024-09-16T13:00:58.622085+0000","last_peered":"2024-09-16T13:00:58.622085+0000","last_clean":"2024-09-16T13:00:58.622085+0000","last_became_active":"2024-09-16T12:59:40.992958+0000","last_became_peered":"2024-09-16T12:59:40.992958+0000","last_unstale":"2024-09-16T13:00:58.622085+0000","last_undegraded":"2024-09-16T13:00:58.622085+0000","last_fullsized":"2024-09-16T13:00:58.622085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:44.409493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"113'593","reported_seq":6823,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.716186+0000","last_change":"2024-09-16T13:00:59.716630+0000","last_active":"2024-09-16T13:00:59.716186+0000","last_peered":"2024-09-16T13:00:59.716186+0000","last_clean":"2024-09-16T13:00:59.716186+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:00:59.716186+0000","last_undegraded":"2024-09-16T13:00:59.716186+0000","last_fullsized":"2024-09-16T13:00:59.716186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015495544,"stat_sum":{"num_bytes":34190854,"num_objects":93,"num_object_clones":17,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":55,"num_read":8673,"num_read_kb":16500,"num_write":545,"num_write_kb":6589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.654810+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:00:58.654810+0000","last_peered":"2024-09-16T13:00:58.654810+0000","last_clean":"2024-09-16T13:00:58.654810+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:00:58.654810+0000","last_undegraded":"2024-09-16T13:00:58.654810+0000","last_fullsized":"2024-09-16T13:00:58.654810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"112'493","reported_seq":2608,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.702214+0000","last_change":"2024-09-16T13:00:59.702434+0000","last_active":"2024-09-16T13:00:59.702214+0000","last_peered":"2024-09-16T13:00:59.702214+0000","last_clean":"2024-09-16T13:00:59.702214+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:00:59.702214+0000","last_undegraded":"2024-09-16T13:00:59.702214+0000","last_fullsized":"2024-09-16T13:00:59.702214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088051009999999992,"stat_sum":{"num_bytes":10577054,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":28,"num_read":2845,"num_read_kb":5421,"num_write":440,"num_write_kb":6163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.623200+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:00:58.623200+0000","last_peered":"2024-09-16T13:00:58.623200+0000","last_clean":"2024-09-16T13:00:58.623200+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:00:58.623200+0000","last_undegraded":"2024-09-16T13:00:58.623200+0000","last_fullsized":"2024-09-16T13:00:58.623200+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"107'491","reported_seq":16659,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.623090+0000","last_change":"2024-09-16T13:00:58.623090+0000","last_active":"2024-09-16T13:00:58.623090+0000","last_peered":"2024-09-16T13:00:58.623090+0000","last_clean":"2024-09-16T13:00:58.623090+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:00:58.623090+0000","last_undegraded":"2024-09-16T13:00:58.623090+0000","last_fullsized":"2024-09-16T13:00:58.623090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081081700000000003,"stat_sum":{"num_bytes":22179795,"num_objects":73,"num_object_clones":10,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":41,"num_read":17779,"num_read_kb":19876,"num_write":599,"num_write_kb":8612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"3.3","version":"0'0","reported_seq":66,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.535269+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:00:58.535269+0000","last_peered":"2024-09-16T13:00:58.535269+0000","last_clean":"2024-09-16T13:00:58.535269+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:00:58.535269+0000","last_undegraded":"2024-09-16T13:00:58.535269+0000","last_fullsized":"2024-09-16T13:00:58.535269+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":132,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.688295+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:00:58.688295+0000","last_peered":"2024-09-16T13:00:58.688295+0000","last_clean":"2024-09-16T13:00:58.688295+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:00:58.688295+0000","last_undegraded":"2024-09-16T13:00:58.688295+0000","last_fullsized":"2024-09-16T13:00:58.688295+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"112'406","reported_seq":17683,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.698109+0000","last_change":"2024-09-16T13:00:59.698603+0000","last_active":"2024-09-16T13:00:59.698109+0000","last_peered":"2024-09-16T13:00:59.698109+0000","last_clean":"2024-09-16T13:00:59.698109+0000","last_became_active":"2024-09-16T12:59:56.105790+0000","last_became_peered":"2024-09-16T12:59:56.105790+0000","last_unstale":"2024-09-16T13:00:59.698109+0000","last_undegraded":"2024-09-16T13:00:59.698109+0000","last_fullsized":"2024-09-16T13:00:59.698109+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:32.594440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010655409999999999,"stat_sum":{"num_bytes":21270591,"num_objects":73,"num_object_clones":15,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":35,"num_read":19018,"num_read_kb":22665,"num_write":445,"num_write_kb":10611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"3.0","version":"0'0","reported_seq":63,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622206+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:00:58.622206+0000","last_peered":"2024-09-16T13:00:58.622206+0000","last_clean":"2024-09-16T13:00:58.622206+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:00:58.622206+0000","last_undegraded":"2024-09-16T13:00:58.622206+0000","last_fullsized":"2024-09-16T13:00:58.622206+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.690125+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:00:58.690125+0000","last_peered":"2024-09-16T13:00:58.690125+0000","last_clean":"2024-09-16T13:00:58.690125+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:00:58.690125+0000","last_undegraded":"2024-09-16T13:00:58.690125+0000","last_fullsized":"2024-09-16T13:00:58.690125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"108'336","reported_seq":2491,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622646+0000","last_change":"2024-09-16T13:00:58.622646+0000","last_active":"2024-09-16T13:00:58.622646+0000","last_peered":"2024-09-16T13:00:58.622646+0000","last_clean":"2024-09-16T13:00:58.622646+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:00:58.622646+0000","last_undegraded":"2024-09-16T13:00:58.622646+0000","last_fullsized":"2024-09-16T13:00:58.622646+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038361599999999998,"stat_sum":{"num_bytes":55265774,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":2772,"num_read_kb":19403,"num_write":382,"num_write_kb":9400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"3.1","version":"55'1","reported_seq":110,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.705553+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:00:58.705553+0000","last_peered":"2024-09-16T13:00:58.705553+0000","last_clean":"2024-09-16T13:00:58.705553+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:00:58.705553+0000","last_undegraded":"2024-09-16T13:00:58.705553+0000","last_fullsized":"2024-09-16T13:00:58.705553+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.654382+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:00:58.654382+0000","last_peered":"2024-09-16T13:00:58.654382+0000","last_clean":"2024-09-16T13:00:58.654382+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:00:58.654382+0000","last_undegraded":"2024-09-16T13:00:58.654382+0000","last_fullsized":"2024-09-16T13:00:58.654382+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"112'340","reported_seq":6950,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.702927+0000","last_change":"2024-09-16T13:00:59.703679+0000","last_active":"2024-09-16T13:00:59.702927+0000","last_peered":"2024-09-16T13:00:59.702927+0000","last_clean":"2024-09-16T13:00:59.702927+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:00:59.702927+0000","last_undegraded":"2024-09-16T13:00:59.702927+0000","last_fullsized":"2024-09-16T13:00:59.702927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096220309999999996,"stat_sum":{"num_bytes":33648800,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":34,"num_read":6872,"num_read_kb":14065,"num_write":282,"num_write_kb":7389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"3.2","version":"107'3","reported_seq":99,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.692631+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:00:58.692631+0000","last_peered":"2024-09-16T13:00:58.692631+0000","last_clean":"2024-09-16T13:00:58.692631+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:00:58.692631+0000","last_undegraded":"2024-09-16T13:00:58.692631+0000","last_fullsized":"2024-09-16T13:00:58.692631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":105,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.688464+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:00:58.688464+0000","last_peered":"2024-09-16T13:00:58.688464+0000","last_clean":"2024-09-16T13:00:58.688464+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:00:58.688464+0000","last_undegraded":"2024-09-16T13:00:58.688464+0000","last_fullsized":"2024-09-16T13:00:58.688464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":192,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.654400+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:00:58.654400+0000","last_peered":"2024-09-16T13:00:58.654400+0000","last_clean":"2024-09-16T13:00:58.654400+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:00:58.654400+0000","last_undegraded":"2024-09-16T13:00:58.654400+0000","last_fullsized":"2024-09-16T13:00:58.654400+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"107'357","reported_seq":3421,"reported_epoch":113,"state":"peering","last_fresh":"2024-09-16T13:00:59.701726+0000","last_change":"2024-09-16T13:00:59.702400+0000","last_active":"2024-09-16T13:00:59.701670+0000","last_peered":"2024-09-16T13:00:59.280360+0000","last_clean":"2024-09-16T13:00:18.737959+0000","last_became_active":"2024-09-16T13:00:57.780795+0000","last_became_peered":"2024-09-16T13:00:57.780795+0000","last_unstale":"2024-09-16T13:00:59.701726+0000","last_undegraded":"2024-09-16T13:00:59.701726+0000","last_fullsized":"2024-09-16T13:00:59.701726+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:27.139439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046786199999999999,"stat_sum":{"num_bytes":20986518,"num_objects":96,"num_object_clones":21,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":51,"num_read":3371,"num_read_kb":12789,"num_write":338,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"113'886","reported_seq":18349,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.700190+0000","last_change":"2024-09-16T13:00:59.700418+0000","last_active":"2024-09-16T13:00:59.700190+0000","last_peered":"2024-09-16T13:00:59.700190+0000","last_clean":"2024-09-16T13:00:59.700190+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:00:59.700190+0000","last_undegraded":"2024-09-16T13:00:59.700190+0000","last_fullsized":"2024-09-16T13:00:59.700190+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0074538599999999997,"stat_sum":{"num_bytes":62057009,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":34,"num_read":20873,"num_read_kb":26853,"num_write":1232,"num_write_kb":13855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.9","version":"111'283","reported_seq":591,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622032+0000","last_change":"2024-09-16T13:00:58.534714+0000","last_active":"2024-09-16T13:00:58.622032+0000","last_peered":"2024-09-16T13:00:58.622032+0000","last_clean":"2024-09-16T13:00:58.622032+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:00:58.622032+0000","last_undegraded":"2024-09-16T13:00:58.622032+0000","last_fullsized":"2024-09-16T13:00:58.622032+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597000000000004,"stat_sum":{"num_bytes":14213462,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":29,"num_read":2910,"num_read_kb":6193,"num_write":280,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.a","version":"112'1216","reported_seq":6513,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.697511+0000","last_change":"2024-09-16T13:00:59.700485+0000","last_active":"2024-09-16T13:00:59.697511+0000","last_peered":"2024-09-16T13:00:59.697511+0000","last_clean":"2024-09-16T13:00:59.697511+0000","last_became_active":"2024-09-16T12:59:44.028122+0000","last_became_peered":"2024-09-16T12:59:44.028122+0000","last_unstale":"2024-09-16T13:00:59.697511+0000","last_undegraded":"2024-09-16T13:00:59.697511+0000","last_fullsized":"2024-09-16T13:00:59.697511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:27:27.517524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031657550000000001,"stat_sum":{"num_bytes":48241999,"num_objects":68,"num_object_clones":12,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":10952,"num_read_kb":20190,"num_write":1917,"num_write_kb":5279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33178,"num_keys_recovered":22,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.b","version":"108'879","reported_seq":6320,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.693493+0000","last_change":"2024-09-16T13:00:59.695017+0000","last_active":"2024-09-16T13:00:59.693493+0000","last_peered":"2024-09-16T13:00:59.693493+0000","last_clean":"2024-09-16T13:00:59.693493+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:00:59.693493+0000","last_undegraded":"2024-09-16T13:00:59.693493+0000","last_fullsized":"2024-09-16T13:00:59.693493+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016754249999999999,"stat_sum":{"num_bytes":24054249,"num_objects":80,"num_object_clones":11,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":40,"num_read":8650,"num_read_kb":19953,"num_write":1333,"num_write_kb":7202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.c","version":"112'471","reported_seq":4985,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.702965+0000","last_change":"2024-09-16T13:00:58.702965+0000","last_active":"2024-09-16T13:00:58.702965+0000","last_peered":"2024-09-16T13:00:58.702965+0000","last_clean":"2024-09-16T13:00:58.702965+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:00:58.702965+0000","last_undegraded":"2024-09-16T13:00:58.702965+0000","last_fullsized":"2024-09-16T13:00:58.702965+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0046687200000000003,"stat_sum":{"num_bytes":42575532,"num_objects":75,"num_object_clones":15,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":41,"num_read":5615,"num_read_kb":15993,"num_write":468,"num_write_kb":7000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.d","version":"112'358","reported_seq":3973,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.750496+0000","last_change":"2024-09-16T13:00:58.702329+0000","last_active":"2024-09-16T13:00:58.750496+0000","last_peered":"2024-09-16T13:00:58.750496+0000","last_clean":"2024-09-16T13:00:58.750496+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:00:58.750496+0000","last_undegraded":"2024-09-16T13:00:58.750496+0000","last_fullsized":"2024-09-16T13:00:58.750496+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.008759434,"stat_sum":{"num_bytes":41233593,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":4804,"num_read_kb":14842,"num_write":366,"num_write_kb":2457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.e","version":"107'768","reported_seq":3580,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:55.031766+0000","last_change":"2024-09-16T13:00:20.279463+0000","last_active":"2024-09-16T13:00:55.031766+0000","last_peered":"2024-09-16T13:00:55.031766+0000","last_clean":"2024-09-16T13:00:18.910076+0000","last_became_active":"2024-09-16T13:00:20.279463+0000","last_became_peered":"2024-09-16T13:00:20.279463+0000","last_unstale":"2024-09-16T13:00:55.031766+0000","last_undegraded":"2024-09-16T13:00:20.278840+0000","last_fullsized":"2024-09-16T13:00:20.278665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:46.024924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637831,"stat_sum":{"num_bytes":40749664,"num_objects":76,"num_object_clones":15,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":36,"num_read":5161,"num_read_kb":7234,"num_write":928,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":76}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"112'798","reported_seq":36283,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.783284+0000","last_change":"2024-09-16T13:00:58.622664+0000","last_active":"2024-09-16T13:00:58.783284+0000","last_peered":"2024-09-16T13:00:58.783284+0000","last_clean":"2024-09-16T13:00:58.783284+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:00:58.783284+0000","last_undegraded":"2024-09-16T13:00:58.783284+0000","last_fullsized":"2024-09-16T13:00:58.783284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042132200000000002,"stat_sum":{"num_bytes":55513432,"num_objects":92,"num_object_clones":17,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":45,"num_read":38796,"num_read_kb":66488,"num_write":1020,"num_write_kb":23487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"28","length":"2"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1220515646,"num_objects":2355,"num_object_clones":409,"num_object_copies":4710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2355,"num_whiteouts":1249,"num_read":327158,"num_read_kb":642349,"num_write":19071,"num_write_kb":307426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":769,"num_bytes_recovered":24015497,"num_keys_recovered":147,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41259008,"data_stored":331052558,"data_compressed":8778922,"data_compressed_allocated":22798336,"data_compressed_original":316408262,"omap_allocated":9108,"internal_metadata":0},"log_size":16296,"ondisk_log_size":16296,"up":61,"acting":61,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304484,"kb_used_data":7116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067356,"statfs":{"total":96636764160,"available":96324972544,"internally_reserved":0,"allocated":7286784,"data_stored":51177415,"data_compressed":1564941,"data_compressed_allocated":3379200,"data_compressed_original":48420106,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.876,"15min":0.90800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.131,"5min":1.393,"15min":1.393},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.88700000000000001,"15min":0.91400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.624}]},{"osd":1,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.90100000000000002,"15min":0.92300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.216,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.024,"15min":0.97599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":8.2279999999999998,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":0.73199999999999998}]},{"osd":2,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.90600000000000003,"15min":0.93600000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3300000000000001,"5min":1.518,"15min":1.518},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92200000000000004,"15min":0.94099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.67700000000000005}]},{"osd":3,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.0069999999999999,"15min":0.96299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.1379999999999999,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":7.2069999999999999,"5min":7.2069999999999999,"15min":7.2069999999999999},"last":1.006}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.98099999999999998,"15min":0.996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.419,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.131,"15min":1.173},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2869999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.94799999999999995}]},{"osd":7,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":1.1799999999999999,"15min":1.24},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2230000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.1819999999999999,"15min":1.216},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4399999999999999,"5min":1.726,"15min":1.726},"last":1.2330000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":4948,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":5066752,"data_stored":28203389,"data_compressed":637556,"data_compressed_allocated":1888256,"data_compressed_original":26017374,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.224,"5min":1.272,"15min":1.272},"last":1.107},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2130000000000001,"5min":1.391,"15min":1.391},"last":0.82899999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":0.96899999999999997,"15min":0.89100000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.275,"5min":14.275,"15min":14.275},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.84299999999999997}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.999,"5min":1.095,"15min":1.111},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.097,"15min":1.075},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.81000000000000005}]},{"osd":4,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0089999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":0.93899999999999995}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.1160000000000001,"15min":1.1499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4399999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.754},{"interface":"front","average":{"1min":1.024,"5min":1.224,"15min":1.258},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.53,"5min":1.748,"15min":1.748},"last":1.0640000000000001}]}]},{"osd":4,"up_from":108,"seq":463856467970,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305132,"kb_used_data":6040,"kb_used_omap":35,"kb_used_meta":299036,"kb_avail":94066708,"statfs":{"total":96636764160,"available":96324308992,"internally_reserved":0,"allocated":6184960,"data_stored":46916818,"data_compressed":1183815,"data_compressed_allocated":2789376,"data_compressed_original":44567026,"omap_allocated":36294,"internal_metadata":306213434},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319872,"kb_used_data":10028,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94051968,"statfs":{"total":96636764160,"available":96309215232,"internally_reserved":0,"allocated":10268672,"data_stored":75985632,"data_compressed":1746327,"data_compressed_allocated":5136384,"data_compressed_original":72309460,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.33900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.74299999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76900000000000002,"15min":0.77800000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.66200000000000003}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.82599999999999996,"15min":0.80500000000000005},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.69899999999999995}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.494}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1319999999999999,"15min":1.1359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":1.327},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.097,"15min":1.115},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.0329999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":1.456},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.516}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.113,"15min":1.1279999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.208,"15min":1.1919999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.6040000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313996,"kb_used_data":4160,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94057844,"statfs":{"total":96636764160,"available":96315232256,"internally_reserved":0,"allocated":4259840,"data_stored":23007206,"data_compressed":723696,"data_compressed_allocated":1552384,"data_compressed_original":21218346,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.80000000000000004,"15min":0.75900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.654,"5min":10.654,"15min":10.654},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.244,"5min":0.84599999999999997,"15min":0.78000000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.60399999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.88400000000000001,"15min":0.86199999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":0.90700000000000003,"15min":0.80900000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":18.117000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.88300000000000001,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.75}]},{"osd":4,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.145},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.1220000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.159,"15min":1.2150000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.415,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.878,"5min":1.1180000000000001,"15min":1.1579999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5509999999999999,"5min":2.125,"15min":2.125},"last":0.89600000000000002}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.96899999999999997}]},{"osd":7,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.2010000000000001,"15min":1.2490000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.792,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.2010000000000001,"15min":1.2609999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3899999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.2290000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301036,"kb_used_data":3672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070804,"statfs":{"total":96636764160,"available":96328503296,"internally_reserved":0,"allocated":3760128,"data_stored":16520810,"data_compressed":541013,"data_compressed_allocated":1187840,"data_compressed_original":14962134,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":0.91300000000000003,"15min":0.85099999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.407,"5min":9.407,"15min":9.407},"last":0.621},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68700000000000006}]},{"osd":2,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.91100000000000003,"15min":0.96299999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.99399999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.496},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.123,"5min":2.254,"15min":2.254},"last":0.53000000000000003}]},{"osd":3,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.80100000000000005,"15min":0.79300000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.3460000000000001,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.997,"15min":1.0429999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.256,"5min":2.044,"15min":2.044},"last":1.016}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.0449999999999999,"15min":1.0740000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4370000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.1819999999999999,"15min":1.222},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.617,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.746}]},{"osd":7,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.163,"15min":1.1879999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6779999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.1319999999999999,"15min":1.167},"min":{"1min":0.438,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.583,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.0629999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":5636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":5771264,"data_stored":33988035,"data_compressed":757468,"data_compressed_allocated":2355200,"data_compressed_original":31719754,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.5920000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.2370000000000001,"15min":1.2909999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7470000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.91500000000000004}]},{"osd":1,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.125,"15min":1.1890000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.681,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.115,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.5149999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.071,"15min":1.069},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.2110000000000001,"15min":1.2250000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0449999999999999,"15min":1.083},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2869999999999999,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.2649999999999999,"15min":1.319},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.623,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.3680000000000001}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.7889999999999997,"5min":4.7889999999999997,"15min":4.7889999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.872}]},{"osd":5,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.284,"15min":1.339},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.75,"5min":6.282,"15min":6.282},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.96799999999999997,"15min":0.98799999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.611,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.80500000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322316,"kb_used_data":8320,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94049524,"statfs":{"total":96636764160,"available":96306712576,"internally_reserved":0,"allocated":8519680,"data_stored":54389993,"data_compressed":1367395,"data_compressed_allocated":3883008,"data_compressed_original":51170894,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85099999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77800000000000002}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76600000000000001,"15min":0.78900000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.234,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.86199999999999999,"15min":0.88800000000000001},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.077,"5min":3.024,"15min":3.024},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":1.004,"15min":1.048},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.103,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.91500000000000004}]},{"osd":4,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.014},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.341}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.349,"15min":1.431},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.234,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.071,"15min":1.0880000000000001},"min":{"1min":0.59799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":1.2809999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.9890000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1299999999999999,"15min":1.147},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.3919999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.27,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.0700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2363392,"data_stored":15270438,"data_compressed":531675,"data_compressed_allocated":1150976,"data_compressed_original":14388694,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7196672,"data_stored":53740805,"data_compressed":1367395,"data_compressed_allocated":3883008,"data_compressed_original":51170894,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2809856,"data_stored":22239159,"data_compressed":723696,"data_compressed_allocated":1552384,"data_compressed_original":21218346,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8720384,"data_stored":74779515,"data_compressed":1694375,"data_compressed_allocated":5050368,"data_compressed_original":71854804,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6193152,"data_stored":54745371,"data_compressed":1560501,"data_compressed_allocated":3575808,"data_compressed_original":52646386,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3641344,"data_stored":27015418,"data_compressed":588209,"data_compressed_allocated":1888256,"data_compressed_original":25562718,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5947392,"data_stored":50517411,"data_compressed":1564941,"data_compressed_allocated":3379200,"data_compressed_original":48420106,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4386816,"data_stored":32744441,"data_compressed":748130,"data_compressed_allocated":2318336,"data_compressed_original":31146314,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:01:03.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:01:03.200 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:03.201 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:03.188+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:03.201 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:03.294 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:03.288+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:03.305 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:03.305 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:03.315 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:03.395 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:03.393+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:03.415 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:03.415 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:03.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:03.430 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:03.461 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:03.487 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:03.495 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:03.488+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:03.514 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:01:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:01:03.526 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:03.526 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:03.532 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-16T13:01:02.714165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1221106116,"num_objects":2363,"num_object_clones":409,"num_object_copies":4726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":1252,"num_read":327224,"num_read_kb":642404,"num_write":19175,"num_write_kb":309057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":771,"num_bytes_recovered":24015535,"num_keys_recovered":147,"num_objects_omap":39,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16375,"ondisk_log_size":16375,"up":87,"acting":87,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2472152,"kb_used_data":49920,"kb_used_omap":47,"kb_used_meta":2422032,"kb_avail":752502568,"statfs":{"total":773094113280,"available":770562629632,"internally_reserved":0,"allocated":51118080,"data_stored":330189298,"data_compressed":8522211,"data_compressed_allocated":22171648,"data_compressed_original":310385094,"omap_allocated":48222,"internal_metadata":2480161698},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-47105986,"num_objects":-104,"num_object_clones":-45,"num_object_copies":-208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-104,"num_whiteouts":-37,"num_read":611,"num_read_kb":883,"num_write":127,"num_write_kb":3109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":16103119,"num_keys_recovered":36,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028122"},"pg_stats":[{"pgid":"4.1f","version":"110'282","reported_seq":3987,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.697585+0000","last_change":"2024-09-16T13:00:59.698023+0000","last_active":"2024-09-16T13:00:59.697585+0000","last_peered":"2024-09-16T13:00:59.697585+0000","last_clean":"2024-09-16T13:00:59.697585+0000","last_became_active":"2024-09-16T12:59:11.766798+0000","last_became_peered":"2024-09-16T12:59:11.766798+0000","last_unstale":"2024-09-16T13:00:59.697585+0000","last_undegraded":"2024-09-16T13:00:59.697585+0000","last_fullsized":"2024-09-16T13:00:59.697585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:22.588888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077473400000000001,"stat_sum":{"num_bytes":20966621,"num_objects":62,"num_object_clones":14,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":30,"num_read":3915,"num_read_kb":19318,"num_write":302,"num_write_kb":5430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1e","version":"113'432","reported_seq":16853,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.701881+0000","last_change":"2024-09-16T13:00:59.702141+0000","last_active":"2024-09-16T13:00:59.701881+0000","last_peered":"2024-09-16T13:00:59.701881+0000","last_clean":"2024-09-16T13:00:59.701881+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:00:59.701881+0000","last_undegraded":"2024-09-16T13:00:59.701881+0000","last_fullsized":"2024-09-16T13:00:59.701881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0088928189999999997,"stat_sum":{"num_bytes":47164207,"num_objects":89,"num_object_clones":18,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":46,"num_read":17148,"num_read_kb":19148,"num_write":434,"num_write_kb":9454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1d","version":"113'391","reported_seq":4743,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.719982+0000","last_change":"2024-09-16T13:00:59.720537+0000","last_active":"2024-09-16T13:00:59.719982+0000","last_peered":"2024-09-16T13:00:59.719982+0000","last_clean":"2024-09-16T13:00:59.719982+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:00:59.719982+0000","last_undegraded":"2024-09-16T13:00:59.719982+0000","last_fullsized":"2024-09-16T13:00:59.719982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.019188515999999999,"stat_sum":{"num_bytes":26736354,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":5128,"num_read_kb":15486,"num_write":330,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1c","version":"113'1047","reported_seq":7454,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.723859+0000","last_change":"2024-09-16T13:00:59.723859+0000","last_active":"2024-09-16T13:00:59.723859+0000","last_peered":"2024-09-16T13:00:59.723859+0000","last_clean":"2024-09-16T13:00:59.723859+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:00:59.723859+0000","last_undegraded":"2024-09-16T13:00:59.723859+0000","last_fullsized":"2024-09-16T13:00:59.723859+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.0076727519999999997,"stat_sum":{"num_bytes":23416045,"num_objects":81,"num_object_clones":10,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":45,"num_read":9824,"num_read_kb":15446,"num_write":1401,"num_write_kb":5620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1b","version":"111'385","reported_seq":21486,"reported_epoch":112,"state":"peering","last_fresh":"2024-09-16T13:00:58.693920+0000","last_change":"2024-09-16T13:00:58.690464+0000","last_active":"2024-09-16T13:00:58.689474+0000","last_peered":"2024-09-16T13:00:58.610355+0000","last_clean":"2024-09-16T13:00:18.903627+0000","last_became_active":"2024-09-16T13:00:57.780913+0000","last_became_peered":"2024-09-16T13:00:57.780913+0000","last_unstale":"2024-09-16T13:00:58.693920+0000","last_undegraded":"2024-09-16T13:00:58.693920+0000","last_fullsized":"2024-09-16T13:00:58.693920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:55.299561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002573915,"stat_sum":{"num_bytes":51579248,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":42,"num_read":22373,"num_read_kb":44612,"num_write":369,"num_write_kb":7677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"106'597","reported_seq":6563,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:54.999280+0000","last_change":"2024-09-16T13:00:20.273695+0000","last_active":"2024-09-16T13:00:54.999280+0000","last_peered":"2024-09-16T13:00:54.999280+0000","last_clean":"2024-09-16T13:00:17.071817+0000","last_became_active":"2024-09-16T13:00:20.273695+0000","last_became_peered":"2024-09-16T13:00:20.273695+0000","last_unstale":"2024-09-16T13:00:54.999280+0000","last_undegraded":"2024-09-16T13:00:20.272075+0000","last_fullsized":"2024-09-16T13:00:20.271923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:03.132002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002565964,"stat_sum":{"num_bytes":51437972,"num_objects":90,"num_object_clones":19,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":90,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":42,"num_read":7614,"num_read_kb":16910,"num_write":835,"num_write_kb":9749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":90}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"113'349","reported_seq":5222,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.715344+0000","last_change":"2024-09-16T13:00:59.716099+0000","last_active":"2024-09-16T13:00:59.715344+0000","last_peered":"2024-09-16T13:00:59.715344+0000","last_clean":"2024-09-16T13:00:59.715344+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:00:59.715344+0000","last_undegraded":"2024-09-16T13:00:59.715344+0000","last_fullsized":"2024-09-16T13:00:59.715344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01521717,"stat_sum":{"num_bytes":20728887,"num_objects":71,"num_object_clones":16,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":37,"num_read":4887,"num_read_kb":18652,"num_write":320,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.18","version":"112'361","reported_seq":6111,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.785057+0000","last_change":"2024-09-16T13:00:58.697117+0000","last_active":"2024-09-16T13:00:58.785057+0000","last_peered":"2024-09-16T13:00:58.785057+0000","last_clean":"2024-09-16T13:00:58.785057+0000","last_became_active":"2024-09-16T13:00:58.691043+0000","last_became_peered":"2024-09-16T13:00:58.691043+0000","last_unstale":"2024-09-16T13:00:58.785057+0000","last_undegraded":"2024-09-16T13:00:58.785057+0000","last_fullsized":"2024-09-16T13:00:58.785057+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:32.993895+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0038786659999999998,"stat_sum":{"num_bytes":49330762,"num_objects":78,"num_object_clones":15,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":36,"num_read":6289,"num_read_kb":20336,"num_write":427,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.17","version":"108'780","reported_seq":7239,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.703198+0000","last_change":"2024-09-16T13:00:59.708220+0000","last_active":"2024-09-16T13:00:59.703198+0000","last_peered":"2024-09-16T13:00:59.703198+0000","last_clean":"2024-09-16T13:00:59.703198+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:00:59.703198+0000","last_undegraded":"2024-09-16T13:00:59.703198+0000","last_fullsized":"2024-09-16T13:00:59.703198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007678603,"stat_sum":{"num_bytes":64984009,"num_objects":85,"num_object_clones":12,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":42,"num_read":10635,"num_read_kb":28964,"num_write":967,"num_write_kb":16664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.16","version":"113'405","reported_seq":2744,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.709331+0000","last_change":"2024-09-16T13:00:59.709660+0000","last_active":"2024-09-16T13:00:59.709331+0000","last_peered":"2024-09-16T13:00:59.709331+0000","last_clean":"2024-09-16T13:00:59.709331+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:00:59.709331+0000","last_undegraded":"2024-09-16T13:00:59.709331+0000","last_fullsized":"2024-09-16T13:00:59.709331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016073513000000001,"stat_sum":{"num_bytes":26333234,"num_objects":73,"num_object_clones":10,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":44,"num_read":4424,"num_read_kb":11797,"num_write":357,"num_write_kb":2394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.15","version":"113'434","reported_seq":15841,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.701013+0000","last_change":"2024-09-16T13:00:59.702236+0000","last_active":"2024-09-16T13:00:59.701013+0000","last_peered":"2024-09-16T13:00:59.701013+0000","last_clean":"2024-09-16T13:00:59.701013+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:00:59.701013+0000","last_undegraded":"2024-09-16T13:00:59.701013+0000","last_fullsized":"2024-09-16T13:00:59.701013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0086595639999999998,"stat_sum":{"num_bytes":41039232,"num_objects":69,"num_object_clones":10,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":40,"num_read":16194,"num_read_kb":17487,"num_write":499,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.14","version":"113'477","reported_seq":22835,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.712692+0000","last_change":"2024-09-16T13:00:59.713166+0000","last_active":"2024-09-16T13:00:59.712692+0000","last_peered":"2024-09-16T13:00:59.712692+0000","last_clean":"2024-09-16T13:00:59.712692+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:00:59.712692+0000","last_undegraded":"2024-09-16T13:00:59.712692+0000","last_fullsized":"2024-09-16T13:00:59.712692+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012577595,"stat_sum":{"num_bytes":68334983,"num_objects":80,"num_object_clones":18,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":39,"num_read":23272,"num_read_kb":34870,"num_write":526,"num_write_kb":16253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.13","version":"106'399","reported_seq":5557,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:54.521735+0000","last_change":"2024-09-16T13:00:20.278920+0000","last_active":"2024-09-16T13:00:54.521735+0000","last_peered":"2024-09-16T13:00:54.521735+0000","last_clean":"2024-09-16T13:00:16.244289+0000","last_became_active":"2024-09-16T13:00:20.278920+0000","last_became_peered":"2024-09-16T13:00:20.278920+0000","last_unstale":"2024-09-16T13:00:54.521735+0000","last_undegraded":"2024-09-16T13:00:20.278208+0000","last_fullsized":"2024-09-16T13:00:20.277996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:09.910764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024518000000000001,"stat_sum":{"num_bytes":55427842,"num_objects":71,"num_object_clones":13,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":71,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":33,"num_read":6200,"num_read_kb":27942,"num_write":410,"num_write_kb":42965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":71}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"107'329","reported_seq":2876,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.693420+0000","last_change":"2024-09-16T13:00:59.694422+0000","last_active":"2024-09-16T13:00:59.693420+0000","last_peered":"2024-09-16T13:00:59.693420+0000","last_clean":"2024-09-16T13:00:59.693420+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:00:59.693420+0000","last_undegraded":"2024-09-16T13:00:59.693420+0000","last_fullsized":"2024-09-16T13:00:59.693420+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013516940000000001,"stat_sum":{"num_bytes":39335036,"num_objects":74,"num_object_clones":9,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":52,"num_read":2795,"num_read_kb":14285,"num_write":359,"num_write_kb":3309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.11","version":"113'307","reported_seq":18494,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.711534+0000","last_change":"2024-09-16T13:00:59.712072+0000","last_active":"2024-09-16T13:00:59.711534+0000","last_peered":"2024-09-16T13:00:59.711534+0000","last_clean":"2024-09-16T13:00:59.711534+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:00:59.711534+0000","last_undegraded":"2024-09-16T13:00:59.711534+0000","last_fullsized":"2024-09-16T13:00:59.711534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.012068730999999999,"stat_sum":{"num_bytes":60439342,"num_objects":68,"num_object_clones":9,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":42,"num_read":18391,"num_read_kb":29155,"num_write":369,"num_write_kb":9723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.10","version":"111'289","reported_seq":5201,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.701580+0000","last_change":"2024-09-16T13:00:59.702319+0000","last_active":"2024-09-16T13:00:59.701580+0000","last_peered":"2024-09-16T13:00:59.701580+0000","last_clean":"2024-09-16T13:00:59.701580+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:00:59.701580+0000","last_undegraded":"2024-09-16T13:00:59.701580+0000","last_fullsized":"2024-09-16T13:00:59.701580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023312129999999999,"stat_sum":{"num_bytes":16079758,"num_objects":73,"num_object_clones":10,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":48,"num_read":5031,"num_read_kb":16220,"num_write":291,"num_write_kb":2924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.690056+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T13:00:58.690056+0000","last_peered":"2024-09-16T13:00:58.690056+0000","last_clean":"2024-09-16T13:00:58.690056+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T13:00:58.690056+0000","last_undegraded":"2024-09-16T13:00:58.690056+0000","last_fullsized":"2024-09-16T13:00:58.690056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"112'357","reported_seq":3247,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.698899+0000","last_change":"2024-09-16T13:00:58.699591+0000","last_active":"2024-09-16T13:00:58.698899+0000","last_peered":"2024-09-16T13:00:58.698899+0000","last_clean":"2024-09-16T13:00:58.698899+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:00:58.698899+0000","last_undegraded":"2024-09-16T13:00:58.698899+0000","last_fullsized":"2024-09-16T13:00:58.698899+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062261210000000003,"stat_sum":{"num_bytes":30423788,"num_objects":59,"num_object_clones":6,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":34,"num_read":3947,"num_read_kb":3256,"num_write":300,"num_write_kb":3627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622085+0000","last_change":"2024-09-16T12:59:40.993864+0000","last_active":"2024-09-16T13:00:58.622085+0000","last_peered":"2024-09-16T13:00:58.622085+0000","last_clean":"2024-09-16T13:00:58.622085+0000","last_became_active":"2024-09-16T12:59:40.992958+0000","last_became_peered":"2024-09-16T12:59:40.992958+0000","last_unstale":"2024-09-16T13:00:58.622085+0000","last_undegraded":"2024-09-16T13:00:58.622085+0000","last_fullsized":"2024-09-16T13:00:58.622085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:44.409493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"113'593","reported_seq":6823,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.716186+0000","last_change":"2024-09-16T13:00:59.716630+0000","last_active":"2024-09-16T13:00:59.716186+0000","last_peered":"2024-09-16T13:00:59.716186+0000","last_clean":"2024-09-16T13:00:59.716186+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:00:59.716186+0000","last_undegraded":"2024-09-16T13:00:59.716186+0000","last_fullsized":"2024-09-16T13:00:59.716186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015495544,"stat_sum":{"num_bytes":34190854,"num_objects":93,"num_object_clones":17,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":55,"num_read":8673,"num_read_kb":16500,"num_write":545,"num_write_kb":6589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.654810+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:00:58.654810+0000","last_peered":"2024-09-16T13:00:58.654810+0000","last_clean":"2024-09-16T13:00:58.654810+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:00:58.654810+0000","last_undegraded":"2024-09-16T13:00:58.654810+0000","last_fullsized":"2024-09-16T13:00:58.654810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"112'493","reported_seq":2608,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.702214+0000","last_change":"2024-09-16T13:00:59.702434+0000","last_active":"2024-09-16T13:00:59.702214+0000","last_peered":"2024-09-16T13:00:59.702214+0000","last_clean":"2024-09-16T13:00:59.702214+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:00:59.702214+0000","last_undegraded":"2024-09-16T13:00:59.702214+0000","last_fullsized":"2024-09-16T13:00:59.702214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088051009999999992,"stat_sum":{"num_bytes":10577054,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":28,"num_read":2845,"num_read_kb":5421,"num_write":440,"num_write_kb":6163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.623200+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:00:58.623200+0000","last_peered":"2024-09-16T13:00:58.623200+0000","last_clean":"2024-09-16T13:00:58.623200+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:00:58.623200+0000","last_undegraded":"2024-09-16T13:00:58.623200+0000","last_fullsized":"2024-09-16T13:00:58.623200+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"107'491","reported_seq":16659,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.623090+0000","last_change":"2024-09-16T13:00:58.623090+0000","last_active":"2024-09-16T13:00:58.623090+0000","last_peered":"2024-09-16T13:00:58.623090+0000","last_clean":"2024-09-16T13:00:58.623090+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:00:58.623090+0000","last_undegraded":"2024-09-16T13:00:58.623090+0000","last_fullsized":"2024-09-16T13:00:58.623090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081081700000000003,"stat_sum":{"num_bytes":22179795,"num_objects":73,"num_object_clones":10,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":41,"num_read":17779,"num_read_kb":19876,"num_write":599,"num_write_kb":8612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"3.3","version":"0'0","reported_seq":66,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.535269+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:00:58.535269+0000","last_peered":"2024-09-16T13:00:58.535269+0000","last_clean":"2024-09-16T13:00:58.535269+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:00:58.535269+0000","last_undegraded":"2024-09-16T13:00:58.535269+0000","last_fullsized":"2024-09-16T13:00:58.535269+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":132,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.688295+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:00:58.688295+0000","last_peered":"2024-09-16T13:00:58.688295+0000","last_clean":"2024-09-16T13:00:58.688295+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:00:58.688295+0000","last_undegraded":"2024-09-16T13:00:58.688295+0000","last_fullsized":"2024-09-16T13:00:58.688295+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"112'406","reported_seq":17683,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.698109+0000","last_change":"2024-09-16T13:00:59.698603+0000","last_active":"2024-09-16T13:00:59.698109+0000","last_peered":"2024-09-16T13:00:59.698109+0000","last_clean":"2024-09-16T13:00:59.698109+0000","last_became_active":"2024-09-16T12:59:56.105790+0000","last_became_peered":"2024-09-16T12:59:56.105790+0000","last_unstale":"2024-09-16T13:00:59.698109+0000","last_undegraded":"2024-09-16T13:00:59.698109+0000","last_fullsized":"2024-09-16T13:00:59.698109+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:32.594440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010655409999999999,"stat_sum":{"num_bytes":21270591,"num_objects":73,"num_object_clones":15,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":35,"num_read":19018,"num_read_kb":22665,"num_write":445,"num_write_kb":10611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"3.0","version":"0'0","reported_seq":63,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622206+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:00:58.622206+0000","last_peered":"2024-09-16T13:00:58.622206+0000","last_clean":"2024-09-16T13:00:58.622206+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:00:58.622206+0000","last_undegraded":"2024-09-16T13:00:58.622206+0000","last_fullsized":"2024-09-16T13:00:58.622206+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.690125+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:00:58.690125+0000","last_peered":"2024-09-16T13:00:58.690125+0000","last_clean":"2024-09-16T13:00:58.690125+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:00:58.690125+0000","last_undegraded":"2024-09-16T13:00:58.690125+0000","last_fullsized":"2024-09-16T13:00:58.690125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"108'336","reported_seq":2491,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622646+0000","last_change":"2024-09-16T13:00:58.622646+0000","last_active":"2024-09-16T13:00:58.622646+0000","last_peered":"2024-09-16T13:00:58.622646+0000","last_clean":"2024-09-16T13:00:58.622646+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:00:58.622646+0000","last_undegraded":"2024-09-16T13:00:58.622646+0000","last_fullsized":"2024-09-16T13:00:58.622646+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038361599999999998,"stat_sum":{"num_bytes":55265774,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":2772,"num_read_kb":19403,"num_write":382,"num_write_kb":9400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"3.1","version":"55'1","reported_seq":110,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.705553+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:00:58.705553+0000","last_peered":"2024-09-16T13:00:58.705553+0000","last_clean":"2024-09-16T13:00:58.705553+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:00:58.705553+0000","last_undegraded":"2024-09-16T13:00:58.705553+0000","last_fullsized":"2024-09-16T13:00:58.705553+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.654382+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:00:58.654382+0000","last_peered":"2024-09-16T13:00:58.654382+0000","last_clean":"2024-09-16T13:00:58.654382+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:00:58.654382+0000","last_undegraded":"2024-09-16T13:00:58.654382+0000","last_fullsized":"2024-09-16T13:00:58.654382+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"112'340","reported_seq":6950,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.702927+0000","last_change":"2024-09-16T13:00:59.703679+0000","last_active":"2024-09-16T13:00:59.702927+0000","last_peered":"2024-09-16T13:00:59.702927+0000","last_clean":"2024-09-16T13:00:59.702927+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:00:59.702927+0000","last_undegraded":"2024-09-16T13:00:59.702927+0000","last_fullsized":"2024-09-16T13:00:59.702927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096220309999999996,"stat_sum":{"num_bytes":33648800,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":34,"num_read":6872,"num_read_kb":14065,"num_write":282,"num_write_kb":7389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"3.2","version":"107'3","reported_seq":99,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.692631+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:00:58.692631+0000","last_peered":"2024-09-16T13:00:58.692631+0000","last_clean":"2024-09-16T13:00:58.692631+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:00:58.692631+0000","last_undegraded":"2024-09-16T13:00:58.692631+0000","last_fullsized":"2024-09-16T13:00:58.692631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":105,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.688464+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:00:58.688464+0000","last_peered":"2024-09-16T13:00:58.688464+0000","last_clean":"2024-09-16T13:00:58.688464+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:00:58.688464+0000","last_undegraded":"2024-09-16T13:00:58.688464+0000","last_fullsized":"2024-09-16T13:00:58.688464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":192,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.654400+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:00:58.654400+0000","last_peered":"2024-09-16T13:00:58.654400+0000","last_clean":"2024-09-16T13:00:58.654400+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:00:58.654400+0000","last_undegraded":"2024-09-16T13:00:58.654400+0000","last_fullsized":"2024-09-16T13:00:58.654400+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"107'357","reported_seq":3421,"reported_epoch":113,"state":"peering","last_fresh":"2024-09-16T13:00:59.701726+0000","last_change":"2024-09-16T13:00:59.702400+0000","last_active":"2024-09-16T13:00:59.701670+0000","last_peered":"2024-09-16T13:00:59.280360+0000","last_clean":"2024-09-16T13:00:18.737959+0000","last_became_active":"2024-09-16T13:00:57.780795+0000","last_became_peered":"2024-09-16T13:00:57.780795+0000","last_unstale":"2024-09-16T13:00:59.701726+0000","last_undegraded":"2024-09-16T13:00:59.701726+0000","last_fullsized":"2024-09-16T13:00:59.701726+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:27.139439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046786199999999999,"stat_sum":{"num_bytes":20986518,"num_objects":96,"num_object_clones":21,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":51,"num_read":3371,"num_read_kb":12789,"num_write":338,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"113'886","reported_seq":18349,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.700190+0000","last_change":"2024-09-16T13:00:59.700418+0000","last_active":"2024-09-16T13:00:59.700190+0000","last_peered":"2024-09-16T13:00:59.700190+0000","last_clean":"2024-09-16T13:00:59.700190+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:00:59.700190+0000","last_undegraded":"2024-09-16T13:00:59.700190+0000","last_fullsized":"2024-09-16T13:00:59.700190+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0074538599999999997,"stat_sum":{"num_bytes":62057009,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":34,"num_read":20873,"num_read_kb":26853,"num_write":1232,"num_write_kb":13855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.9","version":"111'283","reported_seq":591,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622032+0000","last_change":"2024-09-16T13:00:58.534714+0000","last_active":"2024-09-16T13:00:58.622032+0000","last_peered":"2024-09-16T13:00:58.622032+0000","last_clean":"2024-09-16T13:00:58.622032+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:00:58.622032+0000","last_undegraded":"2024-09-16T13:00:58.622032+0000","last_fullsized":"2024-09-16T13:00:58.622032+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597000000000004,"stat_sum":{"num_bytes":14213462,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":29,"num_read":2910,"num_read_kb":6193,"num_write":280,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.a","version":"112'1216","reported_seq":6513,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.697511+0000","last_change":"2024-09-16T13:00:59.700485+0000","last_active":"2024-09-16T13:00:59.697511+0000","last_peered":"2024-09-16T13:00:59.697511+0000","last_clean":"2024-09-16T13:00:59.697511+0000","last_became_active":"2024-09-16T12:59:44.028122+0000","last_became_peered":"2024-09-16T12:59:44.028122+0000","last_unstale":"2024-09-16T13:00:59.697511+0000","last_undegraded":"2024-09-16T13:00:59.697511+0000","last_fullsized":"2024-09-16T13:00:59.697511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:27:27.517524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031657550000000001,"stat_sum":{"num_bytes":48241999,"num_objects":68,"num_object_clones":12,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":10952,"num_read_kb":20190,"num_write":1917,"num_write_kb":5279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33178,"num_keys_recovered":22,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.b","version":"108'879","reported_seq":6320,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.693493+0000","last_change":"2024-09-16T13:00:59.695017+0000","last_active":"2024-09-16T13:00:59.693493+0000","last_peered":"2024-09-16T13:00:59.693493+0000","last_clean":"2024-09-16T13:00:59.693493+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:00:59.693493+0000","last_undegraded":"2024-09-16T13:00:59.693493+0000","last_fullsized":"2024-09-16T13:00:59.693493+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016754249999999999,"stat_sum":{"num_bytes":24054249,"num_objects":80,"num_object_clones":11,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":40,"num_read":8650,"num_read_kb":19953,"num_write":1333,"num_write_kb":7202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.c","version":"112'471","reported_seq":4985,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.702965+0000","last_change":"2024-09-16T13:00:58.702965+0000","last_active":"2024-09-16T13:00:58.702965+0000","last_peered":"2024-09-16T13:00:58.702965+0000","last_clean":"2024-09-16T13:00:58.702965+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:00:58.702965+0000","last_undegraded":"2024-09-16T13:00:58.702965+0000","last_fullsized":"2024-09-16T13:00:58.702965+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0046687200000000003,"stat_sum":{"num_bytes":42575532,"num_objects":75,"num_object_clones":15,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":41,"num_read":5615,"num_read_kb":15993,"num_write":468,"num_write_kb":7000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.d","version":"112'358","reported_seq":3973,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.750496+0000","last_change":"2024-09-16T13:00:58.702329+0000","last_active":"2024-09-16T13:00:58.750496+0000","last_peered":"2024-09-16T13:00:58.750496+0000","last_clean":"2024-09-16T13:00:58.750496+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:00:58.750496+0000","last_undegraded":"2024-09-16T13:00:58.750496+0000","last_fullsized":"2024-09-16T13:00:58.750496+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.008759434,"stat_sum":{"num_bytes":41233593,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":4804,"num_read_kb":14842,"num_write":366,"num_write_kb":2457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.e","version":"107'768","reported_seq":3580,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:55.031766+0000","last_change":"2024-09-16T13:00:20.279463+0000","last_active":"2024-09-16T13:00:55.031766+0000","last_peered":"2024-09-16T13:00:55.031766+0000","last_clean":"2024-09-16T13:00:18.910076+0000","last_became_active":"2024-09-16T13:00:20.279463+0000","last_became_peered":"2024-09-16T13:00:20.279463+0000","last_unstale":"2024-09-16T13:00:55.031766+0000","last_undegraded":"2024-09-16T13:00:20.278840+0000","last_fullsized":"2024-09-16T13:00:20.278665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:46.024924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637831,"stat_sum":{"num_bytes":40749664,"num_objects":76,"num_object_clones":15,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":36,"num_read":5161,"num_read_kb":7234,"num_write":928,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":76}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"112'798","reported_seq":36283,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.783284+0000","last_change":"2024-09-16T13:00:58.622664+0000","last_active":"2024-09-16T13:00:58.783284+0000","last_peered":"2024-09-16T13:00:58.783284+0000","last_clean":"2024-09-16T13:00:58.783284+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:00:58.783284+0000","last_undegraded":"2024-09-16T13:00:58.783284+0000","last_fullsized":"2024-09-16T13:00:58.783284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042132200000000002,"stat_sum":{"num_bytes":55513432,"num_objects":92,"num_object_clones":17,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":45,"num_read":38796,"num_read_kb":66488,"num_write":1020,"num_write_kb":23487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"28","length":"2"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1220515646,"num_objects":2355,"num_object_clones":409,"num_object_copies":4710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2355,"num_whiteouts":1249,"num_read":327158,"num_read_kb":642349,"num_write":19071,"num_write_kb":307426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":769,"num_bytes_recovered":24015497,"num_keys_recovered":147,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41259008,"data_stored":331052558,"data_compressed":8778922,"data_compressed_allocated":22798336,"data_compressed_original":316408262,"omap_allocated":9108,"internal_metadata":0},"log_size":16296,"ondisk_log_size":16296,"up":61,"acting":61,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304484,"kb_used_data":7116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067356,"statfs":{"total":96636764160,"available":96324972544,"internally_reserved":0,"allocated":7286784,"data_stored":51177415,"data_compressed":1564941,"data_compressed_allocated":3379200,"data_compressed_original":48420106,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.876,"15min":0.90800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.131,"5min":1.393,"15min":1.393},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.88700000000000001,"15min":0.91400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.624}]},{"osd":1,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.90100000000000002,"15min":0.92300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.216,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.024,"15min":0.97599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":8.2279999999999998,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":0.73199999999999998}]},{"osd":2,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.90600000000000003,"15min":0.93600000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3300000000000001,"5min":1.518,"15min":1.518},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92200000000000004,"15min":0.94099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.67700000000000005}]},{"osd":3,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.0069999999999999,"15min":0.96299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.1379999999999999,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":7.2069999999999999,"5min":7.2069999999999999,"15min":7.2069999999999999},"last":1.006}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.98099999999999998,"15min":0.996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.419,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.131,"15min":1.173},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2869999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.94799999999999995}]},{"osd":7,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":1.1799999999999999,"15min":1.24},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2230000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.1819999999999999,"15min":1.216},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4399999999999999,"5min":1.726,"15min":1.726},"last":1.2330000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":4948,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":5066752,"data_stored":28203389,"data_compressed":637556,"data_compressed_allocated":1888256,"data_compressed_original":26017374,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.224,"5min":1.272,"15min":1.272},"last":1.107},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2130000000000001,"5min":1.391,"15min":1.391},"last":0.82899999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":0.96899999999999997,"15min":0.89100000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.275,"5min":14.275,"15min":14.275},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.84299999999999997}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.999,"5min":1.095,"15min":1.111},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.097,"15min":1.075},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.81000000000000005}]},{"osd":4,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0089999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":0.93899999999999995}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.1160000000000001,"15min":1.1499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4399999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.754},{"interface":"front","average":{"1min":1.024,"5min":1.224,"15min":1.258},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.53,"5min":1.748,"15min":1.748},"last":1.0640000000000001}]}]},{"osd":4,"up_from":108,"seq":463856467970,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305132,"kb_used_data":6040,"kb_used_omap":35,"kb_used_meta":299036,"kb_avail":94066708,"statfs":{"total":96636764160,"available":96324308992,"internally_reserved":0,"allocated":6184960,"data_stored":46916818,"data_compressed":1183815,"data_compressed_allocated":2789376,"data_compressed_original":44567026,"omap_allocated":36294,"internal_metadata":306213434},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319872,"kb_used_data":10028,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94051968,"statfs":{"total":96636764160,"available":96309215232,"internally_reserved":0,"allocated":10268672,"data_stored":75985632,"data_compressed":1746327,"data_compressed_allocated":5136384,"data_compressed_original":72309460,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.33900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.74299999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76900000000000002,"15min":0.77800000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.66200000000000003}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.82599999999999996,"15min":0.80500000000000005},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.69899999999999995}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.494}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1319999999999999,"15min":1.1359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":1.327},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.097,"15min":1.115},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.0329999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":1.456},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.516}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.113,"15min":1.1279999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.208,"15min":1.1919999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.6040000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313996,"kb_used_data":4160,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94057844,"statfs":{"total":96636764160,"available":96315232256,"internally_reserved":0,"allocated":4259840,"data_stored":23007206,"data_compressed":723696,"data_compressed_allocated":1552384,"data_compressed_original":21218346,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.80000000000000004,"15min":0.75900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.654,"5min":10.654,"15min":10.654},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.244,"5min":0.84599999999999997,"15min":0.78000000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.60399999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.88400000000000001,"15min":0.86199999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":0.90700000000000003,"15min":0.80900000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":18.117000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.88300000000000001,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.75}]},{"osd":4,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.145},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.1220000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.159,"15min":1.2150000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.415,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.878,"5min":1.1180000000000001,"15min":1.1579999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5509999999999999,"5min":2.125,"15min":2.125},"last":0.89600000000000002}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.96899999999999997}]},{"osd":7,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.2010000000000001,"15min":1.2490000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.792,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.2010000000000001,"15min":1.2609999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3899999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.2290000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301036,"kb_used_data":3672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070804,"statfs":{"total":96636764160,"available":96328503296,"internally_reserved":0,"allocated":3760128,"data_stored":16520810,"data_compressed":541013,"data_compressed_allocated":1187840,"data_compressed_original":14962134,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":0.91300000000000003,"15min":0.85099999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.407,"5min":9.407,"15min":9.407},"last":0.621},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68700000000000006}]},{"osd":2,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.91100000000000003,"15min":0.96299999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.99399999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.496},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.123,"5min":2.254,"15min":2.254},"last":0.53000000000000003}]},{"osd":3,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.80100000000000005,"15min":0.79300000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.3460000000000001,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.997,"15min":1.0429999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.256,"5min":2.044,"15min":2.044},"last":1.016}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.0449999999999999,"15min":1.0740000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4370000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.1819999999999999,"15min":1.222},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.617,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.746}]},{"osd":7,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.163,"15min":1.1879999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6779999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.1319999999999999,"15min":1.167},"min":{"1min":0.438,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.583,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.0629999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":5636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":5771264,"data_stored":33988035,"data_compressed":757468,"data_compressed_allocated":2355200,"data_compressed_original":31719754,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.5920000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.2370000000000001,"15min":1.2909999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7470000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.91500000000000004}]},{"osd":1,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.125,"15min":1.1890000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.681,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.115,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.5149999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.071,"15min":1.069},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.2110000000000001,"15min":1.2250000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0449999999999999,"15min":1.083},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2869999999999999,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.2649999999999999,"15min":1.319},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.623,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.3680000000000001}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.7889999999999997,"5min":4.7889999999999997,"15min":4.7889999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.872}]},{"osd":5,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.284,"15min":1.339},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.75,"5min":6.282,"15min":6.282},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.96799999999999997,"15min":0.98799999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.611,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.80500000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322316,"kb_used_data":8320,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94049524,"statfs":{"total":96636764160,"available":96306712576,"internally_reserved":0,"allocated":8519680,"data_stored":54389993,"data_compressed":1367395,"data_compressed_allocated":3883008,"data_compressed_original":51170894,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85099999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77800000000000002}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76600000000000001,"15min":0.78900000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.234,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.86199999999999999,"15min":0.88800000000000001},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.077,"5min":3.024,"15min":3.024},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":1.004,"15min":1.048},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.103,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.91500000000000004}]},{"osd":4,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.014},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.341}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.349,"15min":1.431},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.234,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.071,"15min":1.0880000000000001},"min":{"1min":0.59799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":1.2809999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.9890000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1299999999999999,"15min":1.147},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.3919999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.27,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.0700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2363392,"data_stored":15270438,"data_compressed":531675,"data_compressed_allocated":1150976,"data_compressed_original":14388694,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7196672,"data_stored":53740805,"data_compressed":1367395,"data_compressed_allocated":3883008,"data_compressed_original":51170894,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2809856,"data_stored":22239159,"data_compressed":723696,"data_compressed_allocated":1552384,"data_compressed_original":21218346,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8720384,"data_stored":74779515,"data_compressed":1694375,"data_compressed_allocated":5050368,"data_compressed_original":71854804,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6193152,"data_stored":54745371,"data_compressed":1560501,"data_compressed_allocated":3575808,"data_compressed_original":52646386,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3641344,"data_stored":27015418,"data_compressed":588209,"data_compressed_allocated":1888256,"data_compressed_original":25562718,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5947392,"data_stored":50517411,"data_compressed":1564941,"data_compressed_allocated":3379200,"data_compressed_original":48420106,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4386816,"data_stored":32744441,"data_compressed":748130,"data_compressed_allocated":2318336,"data_compressed_original":31146314,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:01:03.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T13:01:03.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:03.538 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:01:03.561 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:03.562 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:03.563 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:03.563 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:03.563 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:03.563 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:03.563 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:03.563 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:01:03.563 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:03.617 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:03.612+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:03.634 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:03.634 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:03.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:03.697 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:03.693+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:03.742 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:03.742 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:03.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:03.799 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:03.792+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:03.892 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:03.892 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:03.899 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:03.892+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:03.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:03.959 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:01:03.959 INFO:teuthology.orchestra.run.smithi042.stdout:{"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 237/4726 objects degraded (5.015%), 3 pgs degraded","count":3},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":149,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1726491655,"num_in_osds":8,"osd_in_since":1726491579,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":40},{"state_name":"active+undersized+degraded","count":3},{"state_name":"peering","count":2}],"num_pgs":45,"num_pools":4,"num_objects":2363,"data_bytes":1221106116,"bytes_used":2531483648,"bytes_avail":770562629632,"bytes_total":773094113280,"inactive_pgs_ratio":0.04444444552063942,"degraded_objects":237,"degraded_total":4726,"degraded_ratio":0.050148116800677107,"recovering_objects_per_sec":118,"recovering_bytes_per_sec":4306399,"recovering_keys_per_sec":9,"num_objects_recovered":442,"num_bytes_recovered":16103119,"num_keys_recovered":36,"read_bytes_sec":241804,"write_bytes_sec":851383,"read_op_per_sec":163,"write_op_per_sec":33},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-16T13:00:46.265319+0000","services":{}},"progress_events":{}} 2024-09-16T13:01:03.973 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-16T13:01:03.973 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:01:04.000 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:03.997+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:04.050 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:04.050 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:04.059 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:04.102 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:04.096+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:04.150 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:04.151 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:04.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:04.205 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:04.200+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:04.264 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:01:04.267 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:01:04.279 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-16T13:01:03.724293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1221106116,"num_objects":2363,"num_object_clones":409,"num_object_copies":4726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":1252,"num_read":327224,"num_read_kb":642404,"num_write":19175,"num_write_kb":309057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":771,"num_bytes_recovered":24015535,"num_keys_recovered":147,"num_objects_omap":39,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16375,"ondisk_log_size":16375,"up":87,"acting":87,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2472152,"kb_used_data":49920,"kb_used_omap":47,"kb_used_meta":2422032,"kb_avail":752502568,"statfs":{"total":773094113280,"available":770562629632,"internally_reserved":0,"allocated":51118080,"data_stored":330189298,"data_compressed":8522211,"data_compressed_allocated":22171648,"data_compressed_original":310385094,"omap_allocated":48222,"internal_metadata":2480161698},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-37016640,"num_objects":-81,"num_object_clones":-37,"num_object_copies":-162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-97,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-81,"num_whiteouts":-30,"num_read":399,"num_read_kb":348,"num_write":65,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.034627"},"pg_stats":[{"pgid":"4.1f","version":"110'282","reported_seq":3987,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.697585+0000","last_change":"2024-09-16T13:00:59.698023+0000","last_active":"2024-09-16T13:00:59.697585+0000","last_peered":"2024-09-16T13:00:59.697585+0000","last_clean":"2024-09-16T13:00:59.697585+0000","last_became_active":"2024-09-16T12:59:11.766798+0000","last_became_peered":"2024-09-16T12:59:11.766798+0000","last_unstale":"2024-09-16T13:00:59.697585+0000","last_undegraded":"2024-09-16T13:00:59.697585+0000","last_fullsized":"2024-09-16T13:00:59.697585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:22.588888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077473400000000001,"stat_sum":{"num_bytes":20966621,"num_objects":62,"num_object_clones":14,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":30,"num_read":3915,"num_read_kb":19318,"num_write":302,"num_write_kb":5430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1e","version":"113'432","reported_seq":16853,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.701881+0000","last_change":"2024-09-16T13:00:59.702141+0000","last_active":"2024-09-16T13:00:59.701881+0000","last_peered":"2024-09-16T13:00:59.701881+0000","last_clean":"2024-09-16T13:00:59.701881+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:00:59.701881+0000","last_undegraded":"2024-09-16T13:00:59.701881+0000","last_fullsized":"2024-09-16T13:00:59.701881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0088928189999999997,"stat_sum":{"num_bytes":47164207,"num_objects":89,"num_object_clones":18,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":46,"num_read":17148,"num_read_kb":19148,"num_write":434,"num_write_kb":9454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1d","version":"113'391","reported_seq":4743,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.719982+0000","last_change":"2024-09-16T13:00:59.720537+0000","last_active":"2024-09-16T13:00:59.719982+0000","last_peered":"2024-09-16T13:00:59.719982+0000","last_clean":"2024-09-16T13:00:59.719982+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:00:59.719982+0000","last_undegraded":"2024-09-16T13:00:59.719982+0000","last_fullsized":"2024-09-16T13:00:59.719982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.019188515999999999,"stat_sum":{"num_bytes":26736354,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":5128,"num_read_kb":15486,"num_write":330,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1c","version":"113'1047","reported_seq":7454,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.723859+0000","last_change":"2024-09-16T13:00:59.723859+0000","last_active":"2024-09-16T13:00:59.723859+0000","last_peered":"2024-09-16T13:00:59.723859+0000","last_clean":"2024-09-16T13:00:59.723859+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:00:59.723859+0000","last_undegraded":"2024-09-16T13:00:59.723859+0000","last_fullsized":"2024-09-16T13:00:59.723859+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.0076727519999999997,"stat_sum":{"num_bytes":23416045,"num_objects":81,"num_object_clones":10,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":45,"num_read":9824,"num_read_kb":15446,"num_write":1401,"num_write_kb":5620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.1b","version":"111'385","reported_seq":21486,"reported_epoch":112,"state":"peering","last_fresh":"2024-09-16T13:00:58.693920+0000","last_change":"2024-09-16T13:00:58.690464+0000","last_active":"2024-09-16T13:00:58.689474+0000","last_peered":"2024-09-16T13:00:58.610355+0000","last_clean":"2024-09-16T13:00:18.903627+0000","last_became_active":"2024-09-16T13:00:57.780913+0000","last_became_peered":"2024-09-16T13:00:57.780913+0000","last_unstale":"2024-09-16T13:00:58.693920+0000","last_undegraded":"2024-09-16T13:00:58.693920+0000","last_fullsized":"2024-09-16T13:00:58.693920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:55.299561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002573915,"stat_sum":{"num_bytes":51579248,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":42,"num_read":22373,"num_read_kb":44612,"num_write":369,"num_write_kb":7677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"106'597","reported_seq":6563,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:54.999280+0000","last_change":"2024-09-16T13:00:20.273695+0000","last_active":"2024-09-16T13:00:54.999280+0000","last_peered":"2024-09-16T13:00:54.999280+0000","last_clean":"2024-09-16T13:00:17.071817+0000","last_became_active":"2024-09-16T13:00:20.273695+0000","last_became_peered":"2024-09-16T13:00:20.273695+0000","last_unstale":"2024-09-16T13:00:54.999280+0000","last_undegraded":"2024-09-16T13:00:20.272075+0000","last_fullsized":"2024-09-16T13:00:20.271923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:03.132002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002565964,"stat_sum":{"num_bytes":51437972,"num_objects":90,"num_object_clones":19,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":90,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":42,"num_read":7614,"num_read_kb":16910,"num_write":835,"num_write_kb":9749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":90}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"113'349","reported_seq":5222,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.715344+0000","last_change":"2024-09-16T13:00:59.716099+0000","last_active":"2024-09-16T13:00:59.715344+0000","last_peered":"2024-09-16T13:00:59.715344+0000","last_clean":"2024-09-16T13:00:59.715344+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:00:59.715344+0000","last_undegraded":"2024-09-16T13:00:59.715344+0000","last_fullsized":"2024-09-16T13:00:59.715344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01521717,"stat_sum":{"num_bytes":20728887,"num_objects":71,"num_object_clones":16,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":37,"num_read":4887,"num_read_kb":18652,"num_write":320,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.18","version":"112'361","reported_seq":6111,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.785057+0000","last_change":"2024-09-16T13:00:58.697117+0000","last_active":"2024-09-16T13:00:58.785057+0000","last_peered":"2024-09-16T13:00:58.785057+0000","last_clean":"2024-09-16T13:00:58.785057+0000","last_became_active":"2024-09-16T13:00:58.691043+0000","last_became_peered":"2024-09-16T13:00:58.691043+0000","last_unstale":"2024-09-16T13:00:58.785057+0000","last_undegraded":"2024-09-16T13:00:58.785057+0000","last_fullsized":"2024-09-16T13:00:58.785057+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:32.993895+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0038786659999999998,"stat_sum":{"num_bytes":49330762,"num_objects":78,"num_object_clones":15,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":36,"num_read":6289,"num_read_kb":20336,"num_write":427,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.17","version":"108'780","reported_seq":7239,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.703198+0000","last_change":"2024-09-16T13:00:59.708220+0000","last_active":"2024-09-16T13:00:59.703198+0000","last_peered":"2024-09-16T13:00:59.703198+0000","last_clean":"2024-09-16T13:00:59.703198+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:00:59.703198+0000","last_undegraded":"2024-09-16T13:00:59.703198+0000","last_fullsized":"2024-09-16T13:00:59.703198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007678603,"stat_sum":{"num_bytes":64984009,"num_objects":85,"num_object_clones":12,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":42,"num_read":10635,"num_read_kb":28964,"num_write":967,"num_write_kb":16664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.16","version":"113'405","reported_seq":2744,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.709331+0000","last_change":"2024-09-16T13:00:59.709660+0000","last_active":"2024-09-16T13:00:59.709331+0000","last_peered":"2024-09-16T13:00:59.709331+0000","last_clean":"2024-09-16T13:00:59.709331+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:00:59.709331+0000","last_undegraded":"2024-09-16T13:00:59.709331+0000","last_fullsized":"2024-09-16T13:00:59.709331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016073513000000001,"stat_sum":{"num_bytes":26333234,"num_objects":73,"num_object_clones":10,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":44,"num_read":4424,"num_read_kb":11797,"num_write":357,"num_write_kb":2394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.15","version":"113'434","reported_seq":15841,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.701013+0000","last_change":"2024-09-16T13:00:59.702236+0000","last_active":"2024-09-16T13:00:59.701013+0000","last_peered":"2024-09-16T13:00:59.701013+0000","last_clean":"2024-09-16T13:00:59.701013+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:00:59.701013+0000","last_undegraded":"2024-09-16T13:00:59.701013+0000","last_fullsized":"2024-09-16T13:00:59.701013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0086595639999999998,"stat_sum":{"num_bytes":41039232,"num_objects":69,"num_object_clones":10,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":40,"num_read":16194,"num_read_kb":17487,"num_write":499,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.14","version":"113'477","reported_seq":22835,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.712692+0000","last_change":"2024-09-16T13:00:59.713166+0000","last_active":"2024-09-16T13:00:59.712692+0000","last_peered":"2024-09-16T13:00:59.712692+0000","last_clean":"2024-09-16T13:00:59.712692+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:00:59.712692+0000","last_undegraded":"2024-09-16T13:00:59.712692+0000","last_fullsized":"2024-09-16T13:00:59.712692+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012577595,"stat_sum":{"num_bytes":68334983,"num_objects":80,"num_object_clones":18,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":39,"num_read":23272,"num_read_kb":34870,"num_write":526,"num_write_kb":16253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.13","version":"106'399","reported_seq":5557,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:54.521735+0000","last_change":"2024-09-16T13:00:20.278920+0000","last_active":"2024-09-16T13:00:54.521735+0000","last_peered":"2024-09-16T13:00:54.521735+0000","last_clean":"2024-09-16T13:00:16.244289+0000","last_became_active":"2024-09-16T13:00:20.278920+0000","last_became_peered":"2024-09-16T13:00:20.278920+0000","last_unstale":"2024-09-16T13:00:54.521735+0000","last_undegraded":"2024-09-16T13:00:20.278208+0000","last_fullsized":"2024-09-16T13:00:20.277996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:09.910764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024518000000000001,"stat_sum":{"num_bytes":55427842,"num_objects":71,"num_object_clones":13,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":71,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":33,"num_read":6200,"num_read_kb":27942,"num_write":410,"num_write_kb":42965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":71}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"107'329","reported_seq":2876,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.693420+0000","last_change":"2024-09-16T13:00:59.694422+0000","last_active":"2024-09-16T13:00:59.693420+0000","last_peered":"2024-09-16T13:00:59.693420+0000","last_clean":"2024-09-16T13:00:59.693420+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:00:59.693420+0000","last_undegraded":"2024-09-16T13:00:59.693420+0000","last_fullsized":"2024-09-16T13:00:59.693420+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013516940000000001,"stat_sum":{"num_bytes":39335036,"num_objects":74,"num_object_clones":9,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":52,"num_read":2795,"num_read_kb":14285,"num_write":359,"num_write_kb":3309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.11","version":"113'307","reported_seq":18494,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.711534+0000","last_change":"2024-09-16T13:00:59.712072+0000","last_active":"2024-09-16T13:00:59.711534+0000","last_peered":"2024-09-16T13:00:59.711534+0000","last_clean":"2024-09-16T13:00:59.711534+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:00:59.711534+0000","last_undegraded":"2024-09-16T13:00:59.711534+0000","last_fullsized":"2024-09-16T13:00:59.711534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.012068730999999999,"stat_sum":{"num_bytes":60439342,"num_objects":68,"num_object_clones":9,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":42,"num_read":18391,"num_read_kb":29155,"num_write":369,"num_write_kb":9723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.10","version":"111'289","reported_seq":5201,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.701580+0000","last_change":"2024-09-16T13:00:59.702319+0000","last_active":"2024-09-16T13:00:59.701580+0000","last_peered":"2024-09-16T13:00:59.701580+0000","last_clean":"2024-09-16T13:00:59.701580+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:00:59.701580+0000","last_undegraded":"2024-09-16T13:00:59.701580+0000","last_fullsized":"2024-09-16T13:00:59.701580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023312129999999999,"stat_sum":{"num_bytes":16079758,"num_objects":73,"num_object_clones":10,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":48,"num_read":5031,"num_read_kb":16220,"num_write":291,"num_write_kb":2924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.690056+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T13:00:58.690056+0000","last_peered":"2024-09-16T13:00:58.690056+0000","last_clean":"2024-09-16T13:00:58.690056+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T13:00:58.690056+0000","last_undegraded":"2024-09-16T13:00:58.690056+0000","last_fullsized":"2024-09-16T13:00:58.690056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"112'357","reported_seq":3247,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.698899+0000","last_change":"2024-09-16T13:00:58.699591+0000","last_active":"2024-09-16T13:00:58.698899+0000","last_peered":"2024-09-16T13:00:58.698899+0000","last_clean":"2024-09-16T13:00:58.698899+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:00:58.698899+0000","last_undegraded":"2024-09-16T13:00:58.698899+0000","last_fullsized":"2024-09-16T13:00:58.698899+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062261210000000003,"stat_sum":{"num_bytes":30423788,"num_objects":59,"num_object_clones":6,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":34,"num_read":3947,"num_read_kb":3256,"num_write":300,"num_write_kb":3627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622085+0000","last_change":"2024-09-16T12:59:40.993864+0000","last_active":"2024-09-16T13:00:58.622085+0000","last_peered":"2024-09-16T13:00:58.622085+0000","last_clean":"2024-09-16T13:00:58.622085+0000","last_became_active":"2024-09-16T12:59:40.992958+0000","last_became_peered":"2024-09-16T12:59:40.992958+0000","last_unstale":"2024-09-16T13:00:58.622085+0000","last_undegraded":"2024-09-16T13:00:58.622085+0000","last_fullsized":"2024-09-16T13:00:58.622085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:44.409493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"113'593","reported_seq":6823,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.716186+0000","last_change":"2024-09-16T13:00:59.716630+0000","last_active":"2024-09-16T13:00:59.716186+0000","last_peered":"2024-09-16T13:00:59.716186+0000","last_clean":"2024-09-16T13:00:59.716186+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:00:59.716186+0000","last_undegraded":"2024-09-16T13:00:59.716186+0000","last_fullsized":"2024-09-16T13:00:59.716186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015495544,"stat_sum":{"num_bytes":34190854,"num_objects":93,"num_object_clones":17,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":55,"num_read":8673,"num_read_kb":16500,"num_write":545,"num_write_kb":6589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.654810+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:00:58.654810+0000","last_peered":"2024-09-16T13:00:58.654810+0000","last_clean":"2024-09-16T13:00:58.654810+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:00:58.654810+0000","last_undegraded":"2024-09-16T13:00:58.654810+0000","last_fullsized":"2024-09-16T13:00:58.654810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"112'493","reported_seq":2608,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.702214+0000","last_change":"2024-09-16T13:00:59.702434+0000","last_active":"2024-09-16T13:00:59.702214+0000","last_peered":"2024-09-16T13:00:59.702214+0000","last_clean":"2024-09-16T13:00:59.702214+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:00:59.702214+0000","last_undegraded":"2024-09-16T13:00:59.702214+0000","last_fullsized":"2024-09-16T13:00:59.702214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088051009999999992,"stat_sum":{"num_bytes":10577054,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":28,"num_read":2845,"num_read_kb":5421,"num_write":440,"num_write_kb":6163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.623200+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:00:58.623200+0000","last_peered":"2024-09-16T13:00:58.623200+0000","last_clean":"2024-09-16T13:00:58.623200+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:00:58.623200+0000","last_undegraded":"2024-09-16T13:00:58.623200+0000","last_fullsized":"2024-09-16T13:00:58.623200+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"107'491","reported_seq":16659,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.623090+0000","last_change":"2024-09-16T13:00:58.623090+0000","last_active":"2024-09-16T13:00:58.623090+0000","last_peered":"2024-09-16T13:00:58.623090+0000","last_clean":"2024-09-16T13:00:58.623090+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:00:58.623090+0000","last_undegraded":"2024-09-16T13:00:58.623090+0000","last_fullsized":"2024-09-16T13:00:58.623090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081081700000000003,"stat_sum":{"num_bytes":22179795,"num_objects":73,"num_object_clones":10,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":41,"num_read":17779,"num_read_kb":19876,"num_write":599,"num_write_kb":8612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"3.3","version":"0'0","reported_seq":66,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.535269+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:00:58.535269+0000","last_peered":"2024-09-16T13:00:58.535269+0000","last_clean":"2024-09-16T13:00:58.535269+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:00:58.535269+0000","last_undegraded":"2024-09-16T13:00:58.535269+0000","last_fullsized":"2024-09-16T13:00:58.535269+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":132,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.688295+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:00:58.688295+0000","last_peered":"2024-09-16T13:00:58.688295+0000","last_clean":"2024-09-16T13:00:58.688295+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:00:58.688295+0000","last_undegraded":"2024-09-16T13:00:58.688295+0000","last_fullsized":"2024-09-16T13:00:58.688295+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"112'406","reported_seq":17683,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.698109+0000","last_change":"2024-09-16T13:00:59.698603+0000","last_active":"2024-09-16T13:00:59.698109+0000","last_peered":"2024-09-16T13:00:59.698109+0000","last_clean":"2024-09-16T13:00:59.698109+0000","last_became_active":"2024-09-16T12:59:56.105790+0000","last_became_peered":"2024-09-16T12:59:56.105790+0000","last_unstale":"2024-09-16T13:00:59.698109+0000","last_undegraded":"2024-09-16T13:00:59.698109+0000","last_fullsized":"2024-09-16T13:00:59.698109+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:32.594440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010655409999999999,"stat_sum":{"num_bytes":21270591,"num_objects":73,"num_object_clones":15,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":35,"num_read":19018,"num_read_kb":22665,"num_write":445,"num_write_kb":10611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"3.0","version":"0'0","reported_seq":63,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622206+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:00:58.622206+0000","last_peered":"2024-09-16T13:00:58.622206+0000","last_clean":"2024-09-16T13:00:58.622206+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:00:58.622206+0000","last_undegraded":"2024-09-16T13:00:58.622206+0000","last_fullsized":"2024-09-16T13:00:58.622206+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.690125+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:00:58.690125+0000","last_peered":"2024-09-16T13:00:58.690125+0000","last_clean":"2024-09-16T13:00:58.690125+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:00:58.690125+0000","last_undegraded":"2024-09-16T13:00:58.690125+0000","last_fullsized":"2024-09-16T13:00:58.690125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"108'336","reported_seq":2491,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622646+0000","last_change":"2024-09-16T13:00:58.622646+0000","last_active":"2024-09-16T13:00:58.622646+0000","last_peered":"2024-09-16T13:00:58.622646+0000","last_clean":"2024-09-16T13:00:58.622646+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:00:58.622646+0000","last_undegraded":"2024-09-16T13:00:58.622646+0000","last_fullsized":"2024-09-16T13:00:58.622646+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038361599999999998,"stat_sum":{"num_bytes":55265774,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":2772,"num_read_kb":19403,"num_write":382,"num_write_kb":9400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"3.1","version":"55'1","reported_seq":110,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.705553+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:00:58.705553+0000","last_peered":"2024-09-16T13:00:58.705553+0000","last_clean":"2024-09-16T13:00:58.705553+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:00:58.705553+0000","last_undegraded":"2024-09-16T13:00:58.705553+0000","last_fullsized":"2024-09-16T13:00:58.705553+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.654382+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:00:58.654382+0000","last_peered":"2024-09-16T13:00:58.654382+0000","last_clean":"2024-09-16T13:00:58.654382+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:00:58.654382+0000","last_undegraded":"2024-09-16T13:00:58.654382+0000","last_fullsized":"2024-09-16T13:00:58.654382+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"112'340","reported_seq":6950,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.702927+0000","last_change":"2024-09-16T13:00:59.703679+0000","last_active":"2024-09-16T13:00:59.702927+0000","last_peered":"2024-09-16T13:00:59.702927+0000","last_clean":"2024-09-16T13:00:59.702927+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:00:59.702927+0000","last_undegraded":"2024-09-16T13:00:59.702927+0000","last_fullsized":"2024-09-16T13:00:59.702927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096220309999999996,"stat_sum":{"num_bytes":33648800,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":34,"num_read":6872,"num_read_kb":14065,"num_write":282,"num_write_kb":7389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"3.2","version":"107'3","reported_seq":99,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.692631+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:00:58.692631+0000","last_peered":"2024-09-16T13:00:58.692631+0000","last_clean":"2024-09-16T13:00:58.692631+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:00:58.692631+0000","last_undegraded":"2024-09-16T13:00:58.692631+0000","last_fullsized":"2024-09-16T13:00:58.692631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":105,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.688464+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:00:58.688464+0000","last_peered":"2024-09-16T13:00:58.688464+0000","last_clean":"2024-09-16T13:00:58.688464+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:00:58.688464+0000","last_undegraded":"2024-09-16T13:00:58.688464+0000","last_fullsized":"2024-09-16T13:00:58.688464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":192,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.654400+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:00:58.654400+0000","last_peered":"2024-09-16T13:00:58.654400+0000","last_clean":"2024-09-16T13:00:58.654400+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:00:58.654400+0000","last_undegraded":"2024-09-16T13:00:58.654400+0000","last_fullsized":"2024-09-16T13:00:58.654400+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"107'357","reported_seq":3421,"reported_epoch":113,"state":"peering","last_fresh":"2024-09-16T13:00:59.701726+0000","last_change":"2024-09-16T13:00:59.702400+0000","last_active":"2024-09-16T13:00:59.701670+0000","last_peered":"2024-09-16T13:00:59.280360+0000","last_clean":"2024-09-16T13:00:18.737959+0000","last_became_active":"2024-09-16T13:00:57.780795+0000","last_became_peered":"2024-09-16T13:00:57.780795+0000","last_unstale":"2024-09-16T13:00:59.701726+0000","last_undegraded":"2024-09-16T13:00:59.701726+0000","last_fullsized":"2024-09-16T13:00:59.701726+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:27.139439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046786199999999999,"stat_sum":{"num_bytes":20986518,"num_objects":96,"num_object_clones":21,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":51,"num_read":3371,"num_read_kb":12789,"num_write":338,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"113'886","reported_seq":18349,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.700190+0000","last_change":"2024-09-16T13:00:59.700418+0000","last_active":"2024-09-16T13:00:59.700190+0000","last_peered":"2024-09-16T13:00:59.700190+0000","last_clean":"2024-09-16T13:00:59.700190+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:00:59.700190+0000","last_undegraded":"2024-09-16T13:00:59.700190+0000","last_fullsized":"2024-09-16T13:00:59.700190+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0074538599999999997,"stat_sum":{"num_bytes":62057009,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":34,"num_read":20873,"num_read_kb":26853,"num_write":1232,"num_write_kb":13855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.9","version":"111'283","reported_seq":591,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.622032+0000","last_change":"2024-09-16T13:00:58.534714+0000","last_active":"2024-09-16T13:00:58.622032+0000","last_peered":"2024-09-16T13:00:58.622032+0000","last_clean":"2024-09-16T13:00:58.622032+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:00:58.622032+0000","last_undegraded":"2024-09-16T13:00:58.622032+0000","last_fullsized":"2024-09-16T13:00:58.622032+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597000000000004,"stat_sum":{"num_bytes":14213462,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":29,"num_read":2910,"num_read_kb":6193,"num_write":280,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.a","version":"112'1216","reported_seq":6513,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.697511+0000","last_change":"2024-09-16T13:00:59.700485+0000","last_active":"2024-09-16T13:00:59.697511+0000","last_peered":"2024-09-16T13:00:59.697511+0000","last_clean":"2024-09-16T13:00:59.697511+0000","last_became_active":"2024-09-16T12:59:44.028122+0000","last_became_peered":"2024-09-16T12:59:44.028122+0000","last_unstale":"2024-09-16T13:00:59.697511+0000","last_undegraded":"2024-09-16T13:00:59.697511+0000","last_fullsized":"2024-09-16T13:00:59.697511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:27:27.517524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031657550000000001,"stat_sum":{"num_bytes":48241999,"num_objects":68,"num_object_clones":12,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":10952,"num_read_kb":20190,"num_write":1917,"num_write_kb":5279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33178,"num_keys_recovered":22,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.b","version":"108'879","reported_seq":6320,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-16T13:00:59.693493+0000","last_change":"2024-09-16T13:00:59.695017+0000","last_active":"2024-09-16T13:00:59.693493+0000","last_peered":"2024-09-16T13:00:59.693493+0000","last_clean":"2024-09-16T13:00:59.693493+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:00:59.693493+0000","last_undegraded":"2024-09-16T13:00:59.693493+0000","last_fullsized":"2024-09-16T13:00:59.693493+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016754249999999999,"stat_sum":{"num_bytes":24054249,"num_objects":80,"num_object_clones":11,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":40,"num_read":8650,"num_read_kb":19953,"num_write":1333,"num_write_kb":7202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"3"}]},{"pgid":"4.c","version":"112'471","reported_seq":4985,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.702965+0000","last_change":"2024-09-16T13:00:58.702965+0000","last_active":"2024-09-16T13:00:58.702965+0000","last_peered":"2024-09-16T13:00:58.702965+0000","last_clean":"2024-09-16T13:00:58.702965+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:00:58.702965+0000","last_undegraded":"2024-09-16T13:00:58.702965+0000","last_fullsized":"2024-09-16T13:00:58.702965+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0046687200000000003,"stat_sum":{"num_bytes":42575532,"num_objects":75,"num_object_clones":15,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":41,"num_read":5615,"num_read_kb":15993,"num_write":468,"num_write_kb":7000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.d","version":"112'358","reported_seq":3973,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.750496+0000","last_change":"2024-09-16T13:00:58.702329+0000","last_active":"2024-09-16T13:00:58.750496+0000","last_peered":"2024-09-16T13:00:58.750496+0000","last_clean":"2024-09-16T13:00:58.750496+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:00:58.750496+0000","last_undegraded":"2024-09-16T13:00:58.750496+0000","last_fullsized":"2024-09-16T13:00:58.750496+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.008759434,"stat_sum":{"num_bytes":41233593,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":4804,"num_read_kb":14842,"num_write":366,"num_write_kb":2457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"2"}]},{"pgid":"4.e","version":"107'768","reported_seq":3580,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T13:00:55.031766+0000","last_change":"2024-09-16T13:00:20.279463+0000","last_active":"2024-09-16T13:00:55.031766+0000","last_peered":"2024-09-16T13:00:55.031766+0000","last_clean":"2024-09-16T13:00:18.910076+0000","last_became_active":"2024-09-16T13:00:20.279463+0000","last_became_peered":"2024-09-16T13:00:20.279463+0000","last_unstale":"2024-09-16T13:00:55.031766+0000","last_undegraded":"2024-09-16T13:00:20.278840+0000","last_fullsized":"2024-09-16T13:00:20.278665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:46.024924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637831,"stat_sum":{"num_bytes":40749664,"num_objects":76,"num_object_clones":15,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":36,"num_read":5161,"num_read_kb":7234,"num_write":928,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":76}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"112'798","reported_seq":36283,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-16T13:00:58.783284+0000","last_change":"2024-09-16T13:00:58.622664+0000","last_active":"2024-09-16T13:00:58.783284+0000","last_peered":"2024-09-16T13:00:58.783284+0000","last_clean":"2024-09-16T13:00:58.783284+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:00:58.783284+0000","last_undegraded":"2024-09-16T13:00:58.783284+0000","last_fullsized":"2024-09-16T13:00:58.783284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042132200000000002,"stat_sum":{"num_bytes":55513432,"num_objects":92,"num_object_clones":17,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":45,"num_read":38796,"num_read_kb":66488,"num_write":1020,"num_write_kb":23487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"28","length":"2"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1220515646,"num_objects":2355,"num_object_clones":409,"num_object_copies":4710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2355,"num_whiteouts":1249,"num_read":327158,"num_read_kb":642349,"num_write":19071,"num_write_kb":307426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":769,"num_bytes_recovered":24015497,"num_keys_recovered":147,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41259008,"data_stored":331052558,"data_compressed":8778922,"data_compressed_allocated":22798336,"data_compressed_original":316408262,"omap_allocated":9108,"internal_metadata":0},"log_size":16296,"ondisk_log_size":16296,"up":61,"acting":61,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304484,"kb_used_data":7116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067356,"statfs":{"total":96636764160,"available":96324972544,"internally_reserved":0,"allocated":7286784,"data_stored":51177415,"data_compressed":1564941,"data_compressed_allocated":3379200,"data_compressed_original":48420106,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.876,"15min":0.90800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.131,"5min":1.393,"15min":1.393},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.88700000000000001,"15min":0.91400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.624}]},{"osd":1,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.90100000000000002,"15min":0.92300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.216,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.024,"15min":0.97599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":8.2279999999999998,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":0.73199999999999998}]},{"osd":2,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.90600000000000003,"15min":0.93600000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3300000000000001,"5min":1.518,"15min":1.518},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92200000000000004,"15min":0.94099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.67700000000000005}]},{"osd":3,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.0069999999999999,"15min":0.96299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.1379999999999999,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":7.2069999999999999,"5min":7.2069999999999999,"15min":7.2069999999999999},"last":1.006}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.98099999999999998,"15min":0.996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.419,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.131,"15min":1.173},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2869999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.94799999999999995}]},{"osd":7,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":1.1799999999999999,"15min":1.24},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2230000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.1819999999999999,"15min":1.216},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4399999999999999,"5min":1.726,"15min":1.726},"last":1.2330000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":4948,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":5066752,"data_stored":28203389,"data_compressed":637556,"data_compressed_allocated":1888256,"data_compressed_original":26017374,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.224,"5min":1.272,"15min":1.272},"last":1.107},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2130000000000001,"5min":1.391,"15min":1.391},"last":0.82899999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":0.96899999999999997,"15min":0.89100000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.275,"5min":14.275,"15min":14.275},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.84299999999999997}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.999,"5min":1.095,"15min":1.111},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.097,"15min":1.075},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.81000000000000005}]},{"osd":4,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0089999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":0.93899999999999995}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.1160000000000001,"15min":1.1499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4399999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.754},{"interface":"front","average":{"1min":1.024,"5min":1.224,"15min":1.258},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.53,"5min":1.748,"15min":1.748},"last":1.0640000000000001}]}]},{"osd":4,"up_from":108,"seq":463856467970,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305132,"kb_used_data":6040,"kb_used_omap":35,"kb_used_meta":299036,"kb_avail":94066708,"statfs":{"total":96636764160,"available":96324308992,"internally_reserved":0,"allocated":6184960,"data_stored":46916818,"data_compressed":1183815,"data_compressed_allocated":2789376,"data_compressed_original":44567026,"omap_allocated":36294,"internal_metadata":306213434},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319872,"kb_used_data":10028,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94051968,"statfs":{"total":96636764160,"available":96309215232,"internally_reserved":0,"allocated":10268672,"data_stored":75985632,"data_compressed":1746327,"data_compressed_allocated":5136384,"data_compressed_original":72309460,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.33900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.74299999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76900000000000002,"15min":0.77800000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.66200000000000003}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.82599999999999996,"15min":0.80500000000000005},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.69899999999999995}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.494}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1319999999999999,"15min":1.1359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":1.327},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.097,"15min":1.115},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.0329999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":1.456},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.516}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.113,"15min":1.1279999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.208,"15min":1.1919999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.6040000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313996,"kb_used_data":4160,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94057844,"statfs":{"total":96636764160,"available":96315232256,"internally_reserved":0,"allocated":4259840,"data_stored":23007206,"data_compressed":723696,"data_compressed_allocated":1552384,"data_compressed_original":21218346,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.80000000000000004,"15min":0.75900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.654,"5min":10.654,"15min":10.654},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.244,"5min":0.84599999999999997,"15min":0.78000000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.60399999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.88400000000000001,"15min":0.86199999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":0.90700000000000003,"15min":0.80900000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":18.117000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.88300000000000001,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.75}]},{"osd":4,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.145},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.1220000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.159,"15min":1.2150000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.415,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.878,"5min":1.1180000000000001,"15min":1.1579999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5509999999999999,"5min":2.125,"15min":2.125},"last":0.89600000000000002}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.96899999999999997}]},{"osd":7,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.2010000000000001,"15min":1.2490000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.792,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.2010000000000001,"15min":1.2609999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3899999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.2290000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301036,"kb_used_data":3672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070804,"statfs":{"total":96636764160,"available":96328503296,"internally_reserved":0,"allocated":3760128,"data_stored":16520810,"data_compressed":541013,"data_compressed_allocated":1187840,"data_compressed_original":14962134,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":0.91300000000000003,"15min":0.85099999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.407,"5min":9.407,"15min":9.407},"last":0.621},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68700000000000006}]},{"osd":2,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.91100000000000003,"15min":0.96299999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.99399999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.496},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.123,"5min":2.254,"15min":2.254},"last":0.53000000000000003}]},{"osd":3,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.80100000000000005,"15min":0.79300000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.3460000000000001,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.997,"15min":1.0429999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.256,"5min":2.044,"15min":2.044},"last":1.016}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.0449999999999999,"15min":1.0740000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4370000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.1819999999999999,"15min":1.222},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.617,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.746}]},{"osd":7,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.163,"15min":1.1879999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6779999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.1319999999999999,"15min":1.167},"min":{"1min":0.438,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.583,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.0629999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303004,"kb_used_data":5636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068836,"statfs":{"total":96636764160,"available":96326488064,"internally_reserved":0,"allocated":5771264,"data_stored":33988035,"data_compressed":757468,"data_compressed_allocated":2355200,"data_compressed_original":31719754,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.5920000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.2370000000000001,"15min":1.2909999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7470000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.91500000000000004}]},{"osd":1,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.125,"15min":1.1890000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.681,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.115,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.5149999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.071,"15min":1.069},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.2110000000000001,"15min":1.2250000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0449999999999999,"15min":1.083},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2869999999999999,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.2649999999999999,"15min":1.319},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.623,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.3680000000000001}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.7889999999999997,"5min":4.7889999999999997,"15min":4.7889999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.872}]},{"osd":5,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.284,"15min":1.339},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.75,"5min":6.282,"15min":6.282},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.96799999999999997,"15min":0.98799999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.611,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.80500000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322316,"kb_used_data":8320,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94049524,"statfs":{"total":96636764160,"available":96306712576,"internally_reserved":0,"allocated":8519680,"data_stored":54389993,"data_compressed":1367395,"data_compressed_allocated":3883008,"data_compressed_original":51170894,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85099999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77800000000000002}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76600000000000001,"15min":0.78900000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.234,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.86199999999999999,"15min":0.88800000000000001},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.077,"5min":3.024,"15min":3.024},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":1.004,"15min":1.048},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.103,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.91500000000000004}]},{"osd":4,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.014},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.341}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.349,"15min":1.431},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.234,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.071,"15min":1.0880000000000001},"min":{"1min":0.59799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":1.2809999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.9890000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1299999999999999,"15min":1.147},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.3919999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.27,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.0700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2363392,"data_stored":15270438,"data_compressed":531675,"data_compressed_allocated":1150976,"data_compressed_original":14388694,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7196672,"data_stored":53740805,"data_compressed":1367395,"data_compressed_allocated":3883008,"data_compressed_original":51170894,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2809856,"data_stored":22239159,"data_compressed":723696,"data_compressed_allocated":1552384,"data_compressed_original":21218346,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8720384,"data_stored":74779515,"data_compressed":1694375,"data_compressed_allocated":5050368,"data_compressed_original":71854804,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6193152,"data_stored":54745371,"data_compressed":1560501,"data_compressed_allocated":3575808,"data_compressed_original":52646386,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3641344,"data_stored":27015418,"data_compressed":588209,"data_compressed_allocated":1888256,"data_compressed_original":25562718,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5947392,"data_stored":50517411,"data_compressed":1564941,"data_compressed_allocated":3379200,"data_compressed_original":48420106,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4386816,"data_stored":32744441,"data_compressed":748130,"data_compressed_allocated":2318336,"data_compressed_original":31146314,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:01:04.303 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:04.296+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:04.306 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:04.306 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:04.315 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:04.404 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:04.401+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:04.417 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:04.417 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:04.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:04.505 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:04.501+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:04.531 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:04.531 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:04.541 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:04.606 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:04.600+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:04.644 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:04.645 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:04.654 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:04.707 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:04.700+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:04.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T13:01:04.771 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:04.771 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:04.781 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:04.807 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:04.800+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:04.886 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:04.886 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:04.897 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:04.908 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:04.905+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:04.993 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:04.993 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:05.003 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:01:05.009 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:05.004+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:05.110 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:05.104+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:05.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:05.148 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:05.157 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:01:05.211 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:05.209+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:05.257 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:05.257 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:05.266 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:01:05.311 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:05.304+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:05.413 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:05.404+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:05.413 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:05.414 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:05.422 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:05.513 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:05.509+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:05.526 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:05.526 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:05.535 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:05.573 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:01:05.614 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:05.609+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:05.684 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:05.684 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:05.693 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:05.714 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:05.713+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:05.725 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:05.750 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T13:01:05.753 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:05.774 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:05.795 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:05.815 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:05.813+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:05.817 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:05.819 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:05.819 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:05.829 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:05.841 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:05.867 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:01:05.890 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:05.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:05.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:05.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:05.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:05.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:05.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:05.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:01:05.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:05.917 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:05.913+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:05.983 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:05.983 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:05.983 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:05.983 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:05.994 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:06.018 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:06.012+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:06.093 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:06.094 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:06.102 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:06.119 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:06.116+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:06.203 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-16T13:01:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.663187057, 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98766548899999995, 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279128 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534648+0000", 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1459999999999999e-06 2024-09-16T13:01:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534655+0000", 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5220000000000004e-06 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534674+0000", 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9717000000000001e-05 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534683+0000", 2024-09-16T13:01:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534759+0000", 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6074000000000004e-05 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534770+0000", 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0665999999999999e-05 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522281+0000", 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98751079600000002 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522312+0000", 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0876000000000001e-05 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.9 109/108 [PushOp(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:head, version: 106'279, data_included: [0~475136], data_size: 475136, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:head@106'279, size: 475136, copy_subset: [0~475136], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:475136, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:56.636603+0000", 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5612302390000004, 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026500030000000002, 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.636603+0000", 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.636603+0000", 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.636604+0000", 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3570000000000001e-06 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.637425+0000", 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082094999999999998 2024-09-16T13:01:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.637479+0000", 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4249000000000002e-05 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.637499+0000", 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9103e-05 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.637611+0000", 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011230600000000001 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.637621+0000", 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8439999999999993e-06 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.639253+0000", 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001632194 2024-09-16T13:01:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.9 109/108 [PushOp(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29, version: 106'278, data_included: [0~688128,802816~10077,884736~295,1950271~32193], data_size: 730693, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29@106'278, size: 1982464, copy_subset: [0~1982464], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1982464, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5561679250000005, 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067875690000000002, 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641666+0000", 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3060000000000001e-06 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644856+0000", 2024-09-16T13:01:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00318995 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644894+0000", 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7382999999999998e-05 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644913+0000", 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9415999999999998e-05 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645084+0000", 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017036899999999999 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645106+0000", 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2887e-05 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.648453+0000", 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003346258 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279212 4.9 4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:head [delete] snapc 29=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:57.882420+0000", 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.3154135149999995, 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00335185, 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279212 2024-09-16T13:01:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882420+0000", 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882420+0000", 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882423+0000", 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4249999999999998e-06 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882434+0000", 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0702e-05 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882458+0000", 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4647999999999998e-05 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882473+0000", 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4693e-05 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882601+0000", 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012783299999999999 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.882779+0000", 2024-09-16T13:01:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000178464 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.883026+0000", 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024698400000000001 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.884874+0000", 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018472880000000001 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.885447+0000", 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057355199999999996 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.885568+0000", 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120567 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:57.885771+0000", 2024-09-16T13:01:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020369400000000001 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279642 4.d 4:b54cfc8a:::rbd_header.11b5a3d495b5:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:01.711059+0000", 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4867744060000003, 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035981020000000002, 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279642 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711059+0000", 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711059+0000", 2024-09-16T13:01:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711062+0000", 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0139999999999999e-06 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711078+0000", 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5988e-05 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711113+0000", 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5355999999999998e-05 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711130+0000", 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6866000000000001e-05 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711242+0000", 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112358 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.711405+0000", 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016311 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.712033+0000", 2024-09-16T13:01:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00062738500000000005 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.713048+0000", 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001015232 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.714543+0000", 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014950180000000001 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.714594+0000", 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1248000000000001e-05 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.714657+0000", 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2527000000000006e-05 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279698 4.d 4:b54cfc8a:::rbd_header.11b5a3d495b5:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:01.790843+0000", 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4069898009999999, 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026359270000000001, 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279698 2024-09-16T13:01:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790843+0000", 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790843+0000", 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790845+0000", 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.81e-06 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790855+0000", 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8830000000000001e-06 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790883+0000", 2024-09-16T13:01:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7552e-05 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790896+0000", 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3540000000000001e-05 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.790988+0000", 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1791000000000003e-05 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.791141+0000", 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000153423 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.791473+0000", 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000331336 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.792534+0000", 2024-09-16T13:01:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010613280000000001 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793355+0000", 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082062699999999999 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793411+0000", 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6697000000000003e-05 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793479+0000", 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7940000000000003e-05 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279700 4.d 4:b54cfc8a:::rbd_header.11b5a3d495b5:head [watch unwatch cookie 140717156667040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:01.793751+0000", 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4040824870000002, 2024-09-16T13:01:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025856249999999998, 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279700 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793751+0000", 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793751+0000", 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793752+0000", 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.296e-06 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793757+0000", 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793790+0000", 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2265999999999999e-05 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.793797+0000", 2024-09-16T13:01:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5129999999999999e-06 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794104+0000", 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030651300000000001 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794315+0000", 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021124499999999999 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.794580+0000", 2024-09-16T13:01:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026477400000000001 2024-09-16T13:01:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.795644+0000", 2024-09-16T13:01:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010640809999999999 2024-09-16T13:01:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.796164+0000", 2024-09-16T13:01:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00051998500000000004 2024-09-16T13:01:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.796219+0000", 2024-09-16T13:01:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4975000000000001e-05 2024-09-16T13:01:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:01.796336+0000", 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011743699999999999 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279793 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.744160+0000", 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.4536727219999999, 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004380274, 2024-09-16T13:01:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279793 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744160+0000", 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744160+0000", 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744163+0000", 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.067e-06 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744176+0000", 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2184999999999999e-05 2024-09-16T13:01:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744207+0000", 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1754000000000001e-05 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744219+0000", 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1457999999999999e-05 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744341+0000", 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122101 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744351+0000", 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0222e-05 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.748076+0000", 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037249679999999999 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.748277+0000", 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020039299999999999 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.748541+0000", 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000264126 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279845 4.1b 4:d9fcaac3:::rbd_id.image_client.0-clone33:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.822177+0000", 2024-09-16T13:01:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3756558860000001, 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034811210000000002, 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279845 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822177+0000", 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822177+0000", 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822181+0000", 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5429999999999998e-06 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822196+0000", 2024-09-16T13:01:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5676999999999999e-05 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822241+0000", 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4082000000000002e-05 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822259+0000", 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8216e-05 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822414+0000", 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015485 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822684+0000", 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027027000000000003 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.822951+0000", 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026715000000000002 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.824738+0000", 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017868440000000001 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.825411+0000", 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067339799999999997 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.825582+0000", 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017053899999999999 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.825658+0000", 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6551999999999995e-05 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279882 4.c 4:3656acfa:::rbd_data.11b53ee6e6bb.0000000000000009:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.329190761, 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004921407, 2024-09-16T13:01:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279882 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868645+0000", 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5550000000000001e-06 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868656+0000", 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868691+0000", 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4459999999999999e-05 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868703+0000", 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2556e-05 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868818+0000", 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011502000000000001 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.869209+0000", 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039085699999999999 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.869563+0000", 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035391699999999998 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.871027+0000", 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00146393 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873450+0000", 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002422914 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873527+0000", 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7286000000000005e-05 2024-09-16T13:01:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873564+0000", 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6553000000000001e-05 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279890 4.e 4:726a396f:::rbd_data.11b53ee6e6bb.0000000000000017:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.873597+0000", 2024-09-16T13:01:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3242363539999999, 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028654750000000001, 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279890 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873597+0000", 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873597+0000", 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873599+0000", 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3140000000000002e-06 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873611+0000", 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1491000000000001e-05 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873633+0000", 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2152000000000001e-05 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873644+0000", 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1664999999999999e-05 2024-09-16T13:01:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873734+0000", 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9480000000000004e-05 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873966+0000", 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000231695 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.874176+0000", 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021012299999999999 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.875582+0000", 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001406205 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876374+0000", 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00079198300000000001 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876423+0000", 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8875999999999998e-05 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876462+0000", 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9490999999999998e-05 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279895 4.1 4:848811f6:::rbd_data.11b53ee6e6bb.000000000000003c:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.876321+0000", 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3215122949999998, 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003317801, 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279895 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876321+0000", 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876321+0000", 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876322+0000", 2024-09-16T13:01:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5719999999999999e-06 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876330+0000", 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8720000000000002e-06 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876351+0000", 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0820999999999999e-05 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876362+0000", 2024-09-16T13:01:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0558e-05 2024-09-16T13:01:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876464+0000", 2024-09-16T13:01:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102446 2024-09-16T13:01:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.876729+0000", 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533900000000002 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.877078+0000", 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034812899999999997 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.879424+0000", 2024-09-16T13:01:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023464250000000001 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.879500+0000", 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5981000000000003e-05 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.879579+0000", 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8870999999999998e-05 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.879639+0000", 2024-09-16T13:01:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9787000000000002e-05 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279967 4.1 4:8218f49a:::rbd_object_map.11b552946c0f.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.951941+0000", 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2458925590000001, 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025624770000000001, 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279967 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951941+0000", 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951941+0000", 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951942+0000", 2024-09-16T13:01:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.403e-06 2024-09-16T13:01:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951948+0000", 2024-09-16T13:01:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2879999999999998e-06 2024-09-16T13:01:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951969+0000", 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0350999999999999e-05 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.951981+0000", 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2557e-05 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.952140+0000", 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015917500000000001 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.952286+0000", 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014593000000000001 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.952469+0000", 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018234499999999999 2024-09-16T13:01:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.953732+0000", 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012638930000000001 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.954366+0000", 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063367499999999999 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.954445+0000", 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8572000000000006e-05 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.954503+0000", 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8288000000000002e-05 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280527 4.13 4:cc227b3c:::rbd_data.11b5cb8797c6.0000000000000000:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.32470856799999998, 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052047300000000003, 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280527 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873126+0000", 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6530000000000001e-06 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873133+0000", 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0940000000000002e-06 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873153+0000", 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9538000000000002e-05 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873161+0000", 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1009999999999992e-06 2024-09-16T13:01:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873245+0000", 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4075999999999999e-05 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873360+0000", 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114951 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873530+0000", 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016994699999999999 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.877617+0000", 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040873000000000003 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878160+0000", 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054299199999999995 2024-09-16T13:01:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878251+0000", 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1062000000000003e-05 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878329+0000", 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8016000000000007e-05 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280530 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.32464656400000003, 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056839330000000004, 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280530 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8199999999999999e-07 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873191+0000", 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4599999999999999e-06 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873198+0000", 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8399999999999997e-06 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873201+0000", 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.641e-06 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873298+0000", 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6682999999999996e-05 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873451+0000", 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015304000000000001 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873692+0000", 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024124300000000001 2024-09-16T13:01:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.876380+0000", 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026873980000000001 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878771+0000", 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002391013 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878824+0000", 2024-09-16T13:01:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3136000000000001e-05 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878870+0000", 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6796999999999999e-05 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280624 4.1b 4:d8560506:::rbd_header.11b5de89dd5b:head [watch watch cookie 140717155157888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.999647+0000", 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.19818625600000001, 2024-09-16T13:01:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004107388, 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280624 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999647+0000", 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999647+0000", 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999649+0000", 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.48e-06 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999663+0000", 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3623e-05 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999694+0000", 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0729999999999999e-05 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999707+0000", 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3152999999999999e-05 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999847+0000", 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140605 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.000097+0000", 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024959100000000002 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.000584+0000", 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048667799999999999 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.002009+0000", 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001424795 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.003370+0000", 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001361686 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.003520+0000", 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014938199999999999 2024-09-16T13:01:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.003754+0000", 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000234665 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280629 4.1b 4:d8560506:::rbd_header.11b5de89dd5b:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.008552+0000", 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.18928115000000001, 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029307460000000001, 2024-09-16T13:01:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280629 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.008552+0000", 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.008552+0000", 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.008553+0000", 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4100000000000001e-06 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.008560+0000", 2024-09-16T13:01:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4960000000000001e-06 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.008579+0000", 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.91e-05 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.008586+0000", 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.418e-06 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.008683+0000", 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6119999999999995e-05 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.008817+0000", 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013418800000000001 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.009212+0000", 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039575100000000001 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.010377+0000", 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001164251 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.011363+0000", 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000986282 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.011422+0000", 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8527999999999999e-05 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.011483+0000", 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1202000000000001e-05 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280639 4.1b 4:d8560506:::rbd_header.11b5de89dd5b:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.019921+0000", 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.17791204299999999, 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027142429999999999, 2024-09-16T13:01:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280639 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.019921+0000", 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.019921+0000", 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.019922+0000", 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.164e-06 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.019928+0000", 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7540000000000003e-06 2024-09-16T13:01:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.019944+0000", 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5528000000000001e-05 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.019949+0000", 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9050000000000002e-06 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.020007+0000", 2024-09-16T13:01:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7980999999999999e-05 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.020111+0000", 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103618 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.020319+0000", 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020815300000000001 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.021307+0000", 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00098829100000000008 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.022496+0000", 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011886550000000001 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.022559+0000", 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2966000000000007e-05 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.022635+0000", 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6228000000000003e-05 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280643 4.1b 4:d8560506:::rbd_header.11b5de89dd5b:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.028019+0000", 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.16981381300000001, 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038646489999999999, 2024-09-16T13:01:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280643 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028019+0000", 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028019+0000", 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028020+0000", 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2799999999999995e-07 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028026+0000", 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5480000000000001e-06 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028057+0000", 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0849000000000002e-05 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028065+0000", 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4749999999999993e-06 2024-09-16T13:01:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028460+0000", 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039461299999999998 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028616+0000", 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015688 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.029138+0000", 2024-09-16T13:01:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052184400000000004 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.030338+0000", 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011994550000000001 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.031691+0000", 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001353618 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.031786+0000", 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5011000000000002e-05 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.031884+0000", 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7528000000000005e-05 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280644 4.1b 4:d8560506:::rbd_header.11b5de89dd5b:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.028588+0000", 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.16924467000000001, 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003593195, 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280644 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028588+0000", 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028588+0000", 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028590+0000", 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.302e-06 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028596+0000", 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4729999999999997e-06 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028608+0000", 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2162e-05 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.028615+0000", 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2820000000000002e-06 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.029906+0000", 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001290846 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.030014+0000", 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108805 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.031944+0000", 2024-09-16T13:01:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001929736 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.032041+0000", 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7157999999999994e-05 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.032182+0000", 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140431 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:06.323 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:06.324 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:06.325 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:06.320+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:06.425 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:06.420+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:06.432 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:06.433 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:06.433 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:06.433 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:06.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:06.526 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:06.521+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:06.543 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:06.543 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:06.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:06.626 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:06.625+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:06.656 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960230824999996, 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960215798999997, 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:01:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960199777, 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:01:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:01:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960183680999997, 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:01:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:01:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960167791000003, 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:01:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:01:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960152121, 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960136374000001, 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:01:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960120586000002, 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:01:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:01:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:01:06.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960105263999999, 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:01:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960085339000003, 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.702 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960069390999998, 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:06.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:01:06.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:01:06.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960054229999997, 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:01:06.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:06.711 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960038582000003, 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:01:06.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:01:06.715 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960023485000001, 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:01:06.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:06.720 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.960008223000003, 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:01:06.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.959992477, 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:01:06.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.959976879999999, 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:06.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:01:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.959961575999998, 2024-09-16T13:01:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:01:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.959946137999999, 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:01:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:01:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:01:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:01:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:01:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.959930425000003, 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:01:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:06.748 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:06.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:06.750 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:06.745+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:06.851 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:06.849+0000 7f866f15f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22337) UID: 0 2024-09-16T13:01:06.853 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:06.853 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:06.853 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:06.853 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:06.863 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:06.952 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:06.944+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:06.966 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:06.967 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:06.967 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:06.967 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:06.967 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:06.977 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:07.053 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:07.048+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:07.079 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:07.079 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:07.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254136 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 136192~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.225610680999999, 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082717923999999998, 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254136 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849492+0000", 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849494+0000", 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0540000000000002e-06 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849500+0000", 2024-09-16T13:01:07.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2899999999999999e-06 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849504+0000", 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8120000000000001e-06 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.931987+0000", 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082483231000000004 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932061+0000", 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4181e-05 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932209+0000", 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014796399999999999 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254137 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 139264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.225594405999999, 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082930443000000006, 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254137 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:07.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:07.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849510+0000", 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0530000000000001e-06 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849516+0000", 2024-09-16T13:01:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8359999999999997e-06 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849520+0000", 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932239+0000", 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082719191999999997 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932307+0000", 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7824000000000007e-05 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932438+0000", 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000131354 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.089 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254138 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 146432~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.225578475999999, 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083100513000000001, 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254138 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7300000000000001e-07 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849526+0000", 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9089999999999998e-06 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849532+0000", 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6420000000000004e-06 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849535+0000", 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7740000000000002e-06 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932467+0000", 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082931469999999993 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932535+0000", 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7815e-05 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932624+0000", 2024-09-16T13:01:07.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9430000000000003e-05 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254139 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 149504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.225562919000001, 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083267068, 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254139 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:07.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849540+0000", 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849542+0000", 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9379999999999999e-06 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849547+0000", 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849551+0000", 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5880000000000002e-06 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932652+0000", 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083101176999999998 2024-09-16T13:01:07.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932719+0000", 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.703e-05 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932806+0000", 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7340000000000001e-05 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254140 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 156672~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.225547645999999, 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083436222000000004, 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254140 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849557+0000", 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8419999999999999e-06 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849564+0000", 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7079999999999997e-06 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849569+0000", 2024-09-16T13:01:07.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.386e-06 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932833+0000", 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083264099999999994 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932900+0000", 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7007000000000004e-05 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932991+0000", 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0815000000000004e-05 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254141 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.225529093999999, 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083599532000000004, 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254141 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849574+0000", 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1699999999999998e-07 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849576+0000", 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849582+0000", 2024-09-16T13:01:07.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9769999999999997e-06 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849585+0000", 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933018+0000", 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083432372000000005 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933086+0000", 2024-09-16T13:01:07.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7862999999999997e-05 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933173+0000", 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7050000000000002e-05 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254142 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:07.107 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.225512943999998, 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083767603999999996, 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254142 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849590+0000", 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.51e-07 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849592+0000", 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0360000000000001e-06 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849598+0000", 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4139999999999999e-06 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849602+0000", 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.659e-06 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933199+0000", 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083597150999999995 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933267+0000", 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7933000000000002e-05 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933357+0000", 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9859999999999997e-05 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254143 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.225495995999999, 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083932426000000004, 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254143 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.112 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849607+0000", 2024-09-16T13:01:07.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849609+0000", 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.165e-06 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849615+0000", 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2029999999999997e-06 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849619+0000", 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933382+0000", 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083763447000000005 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933447+0000", 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4993000000000006e-05 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933539+0000", 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1434000000000006e-05 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254145 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.225479434, 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084130242999999993, 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254145 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851596+0000", 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019732550000000001 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851602+0000", 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5720000000000002e-06 2024-09-16T13:01:07.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851610+0000", 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5529999999999993e-06 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851616+0000", 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6409999999999999e-06 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933580+0000", 2024-09-16T13:01:07.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.081963854000000003 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933653+0000", 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3901999999999998e-05 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933753+0000", 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9465999999999999e-05 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254149 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.223462910999999, 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082661199000000005, 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254149 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851640+0000", 2024-09-16T13:01:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851642+0000", 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851648+0000", 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9200000000000001e-06 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851652+0000", 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0439999999999997e-06 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934105+0000", 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082453495000000002 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934193+0000", 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7433000000000001e-05 2024-09-16T13:01:07.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934300+0000", 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107513 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254151 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.22344129, 2024-09-16T13:01:07.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082910496, 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254151 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1799999999999995e-07 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851664+0000", 2024-09-16T13:01:07.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4399999999999999e-06 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851670+0000", 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3509999999999998e-06 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851674+0000", 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.912e-06 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934385+0000", 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082710396000000005 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934471+0000", 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6040999999999998e-05 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934571+0000", 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100738 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254153 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.223423836999999, 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083065172000000007, 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254153 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851679+0000", 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851681+0000", 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.137e-06 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851687+0000", 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.643e-06 2024-09-16T13:01:07.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851690+0000", 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5580000000000001e-06 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934597+0000", 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082906881000000002 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934663+0000", 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5828999999999996e-05 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934743+0000", 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0714e-05 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254155 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.223408149000001, 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083254350000000005, 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254155 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-16T13:01:07.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851697+0000", 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.187e-06 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851702+0000", 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7520000000000002e-06 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851706+0000", 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7979999999999999e-06 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934769+0000", 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083062834000000002 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934829+0000", 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0430999999999998e-05 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934948+0000", 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118905 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254157 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 207872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.223392224000001, 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083456394000000003, 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254157 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851712+0000", 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0150000000000002e-06 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851718+0000", 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5999999999999997e-06 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851722+0000", 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5810000000000001e-06 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934992+0000", 2024-09-16T13:01:07.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083270799000000006 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935066+0000", 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3851999999999997e-05 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935166+0000", 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100145 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254159 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 210944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.223376878, 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083625606000000005, 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254159 2024-09-16T13:01:07.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.145 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:07.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851726+0000", 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851728+0000", 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851734+0000", 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7429999999999999e-06 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851738+0000", 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9620000000000004e-06 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935195+0000", 2024-09-16T13:01:07.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083457518999999994 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935262+0000", 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6465999999999997e-05 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935351+0000", 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9314000000000006e-05 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.148 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254161 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 218112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.223360887999998, 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083866861000000001, 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254161 2024-09-16T13:01:07.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851742+0000", 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851744+0000", 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1550000000000001e-06 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851749+0000", 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5829999999999997e-06 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851753+0000", 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6990000000000001e-06 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935377+0000", 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083624025000000005 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935454+0000", 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7168000000000004e-05 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935608+0000", 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153774 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254163 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 221184~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.153 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.223345112000001, 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084055344000000004, 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254163 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:07.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851760+0000", 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2249999999999999e-06 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851765+0000", 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7799999999999997e-06 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851769+0000", 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-06 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935654+0000", 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083884829999999994 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935723+0000", 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9163000000000002e-05 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935812+0000", 2024-09-16T13:01:07.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9237e-05 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254164 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.223329068999998, 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084726810999999999, 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.158 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254164 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851775+0000", 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.945e-06 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851781+0000", 2024-09-16T13:01:07.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851785+0000", 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6320000000000001e-06 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936053+0000", 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084268018 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936130+0000", 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7478999999999995e-05 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936500+0000", 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036965099999999997 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254165 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 228352~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.162 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.223313545, 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084960919999999995, 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254165 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:07.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851791+0000", 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2819999999999999e-06 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851797+0000", 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7969999999999997e-06 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851801+0000", 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6679999999999999e-06 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936554+0000", 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084753203999999999 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936625+0000", 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1406e-05 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936750+0000", 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012416700000000001 2024-09-16T13:01:07.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254166 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 6144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.223297733999999, 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.085115273000000005, 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.167 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254166 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856445+0000", 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046400929999999996 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856451+0000", 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5119999999999997e-06 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856463+0000", 2024-09-16T13:01:07.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2252999999999999e-05 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856470+0000", 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4160000000000001e-06 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936767+0000", 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080297258999999996 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936832+0000", 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4685999999999997e-05 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936920+0000", 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8053999999999994e-05 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.171 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:07.173 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:07.173 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:07.168+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:07.273 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:07.269+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:07.283 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:01:07.323 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:07.324 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:07.324 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:07.324 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:07.334 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:07.375 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:07.373+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:07.431 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:07.431 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:07.432 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:07.432 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:07.432 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:07.441 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:07.475 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:07.473+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:07.544 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260177 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463779551999998, 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039279541000000001, 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.546 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260177 2024-09-16T13:01:07.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.546 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:07.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075845+0000", 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075847+0000", 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075852+0000", 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.507e-06 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075856+0000", 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6389999999999998e-06 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.114965+0000", 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039109496000000001 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115034+0000", 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8378000000000002e-05 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115124+0000", 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0334999999999995e-05 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260178 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463764156, 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039530044, 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260178 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075866+0000", 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0909999999999998e-06 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075872+0000", 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.139e-06 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075876+0000", 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0419999999999996e-06 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115194+0000", 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039317193 2024-09-16T13:01:07.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115259+0000", 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5399000000000002e-05 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115390+0000", 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130803 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260179 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463743306000001, 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039926426000000001, 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260179 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.555 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075883+0000", 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6959999999999999e-06 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075889+0000", 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9479999999999996e-06 2024-09-16T13:01:07.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075893+0000", 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7840000000000001e-06 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115419+0000", 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039526517999999997 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115577+0000", 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015783999999999999 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115807+0000", 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023010399999999999 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.559 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260180 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463727558999999, 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040245398000000002, 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260180 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075897+0000", 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:01:07.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075899+0000", 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8169999999999999e-06 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075904+0000", 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075908+0000", 2024-09-16T13:01:07.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5889999999999999e-06 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115837+0000", 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039929193000000002 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115978+0000", 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014115800000000001 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116142+0000", 2024-09-16T13:01:07.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016352900000000001 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260181 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 180224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463712352000002, 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040482388000000001, 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260181 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075914+0000", 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6759999999999999e-06 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075919+0000", 2024-09-16T13:01:07.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2360000000000004e-06 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075922+0000", 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116171+0000", 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040248702999999997 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116239+0000", 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8391000000000003e-05 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116394+0000", 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000154575 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260182 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463697641, 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042150168000000002, 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260182 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075927+0000", 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075929+0000", 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7859999999999999e-06 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075934+0000", 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7679999999999997e-06 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075938+0000", 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2590000000000001e-06 2024-09-16T13:01:07.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116426+0000", 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040488077999999997 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116493+0000", 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7681000000000005e-05 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118077+0000", 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015832260000000001 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260183 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463682652999999, 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040886082999999997, 2024-09-16T13:01:07.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260183 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075942+0000", 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075943+0000", 2024-09-16T13:01:07.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.778e-06 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075949+0000", 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075953+0000", 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116672+0000", 2024-09-16T13:01:07.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040719563 2024-09-16T13:01:07.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116743+0000", 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0729000000000007e-05 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116827+0000", 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4364999999999996e-05 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.578 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260184 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463667489999999, 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041043524999999997, 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260184 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:07.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075959+0000", 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6869999999999999e-06 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075964+0000", 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.242e-06 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075967+0000", 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116850+0000", 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040882977000000001 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116916+0000", 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5413000000000005e-05 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117000+0000", 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4431999999999994e-05 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260185 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463652250999999, 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041177895999999999, 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260185 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075974+0000", 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0049999999999999e-06 2024-09-16T13:01:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075980+0000", 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7470000000000002e-06 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075984+0000", 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117022+0000", 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041038789999999999 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117090+0000", 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7712000000000005e-05 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117150+0000", 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9542e-05 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260257 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463610397, 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041222406000000003, 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260257 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076017+0000", 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4990000000000001e-06 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076024+0000", 2024-09-16T13:01:07.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076028+0000", 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9319999999999998e-06 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117162+0000", 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04113472 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117199+0000", 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6507999999999997e-05 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117236+0000", 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7023000000000001e-05 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260258 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463591877999999, 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041343291999999997, 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260258 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:07.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076035+0000", 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.058e-06 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076041+0000", 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.998e-06 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076044+0000", 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3900000000000002e-06 2024-09-16T13:01:07.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117256+0000", 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041212555999999997 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117323+0000", 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6568000000000004e-05 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117375+0000", 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2361999999999998e-05 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260259 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463576459999999, 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041437587999999997, 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260259 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5499999999999999e-07 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076050+0000", 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076055+0000", 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5609999999999998e-06 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076059+0000", 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117396+0000", 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041336879999999999 2024-09-16T13:01:07.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117454+0000", 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8443999999999998e-05 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117485+0000", 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1047999999999998e-05 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260260 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463561661, 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041496984000000001, 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260260 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076063+0000", 2024-09-16T13:01:07.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076070+0000", 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7919999999999993e-06 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076076+0000", 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9359999999999996e-06 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076080+0000", 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8240000000000001e-06 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117497+0000", 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041416896000000002 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117531+0000", 2024-09-16T13:01:07.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4e-05 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117559+0000", 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8177000000000001e-05 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260261 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:07.606 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463539425, 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041555793000000001, 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260261 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-16T13:01:07.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076087+0000", 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7850000000000001e-06 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076093+0000", 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6879999999999996e-06 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076096+0000", 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117570+0000", 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041474329999999997 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117611+0000", 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0852999999999997e-05 2024-09-16T13:01:07.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117640+0000", 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9213e-05 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260262 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463523979000001, 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041643056999999997, 2024-09-16T13:01:07.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260262 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076102+0000", 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8509999999999999e-06 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076108+0000", 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8560000000000003e-06 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076111+0000", 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0620000000000001e-06 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117651+0000", 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041539980999999997 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117703+0000", 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1975000000000003e-05 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117743+0000", 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9872e-05 2024-09-16T13:01:07.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260263 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463508998000002, 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041724412000000002, 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260263 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076117+0000", 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.762e-06 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076123+0000", 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:01:07.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076126+0000", 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.427e-06 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117757+0000", 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041631230999999998 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117792+0000", 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4078999999999997e-05 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117839+0000", 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7861000000000001e-05 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260264 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:07.620 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463494151999999, 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041783638999999997, 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260264 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:07.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076132+0000", 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.751e-06 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076138+0000", 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6729999999999997e-06 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076141+0000", 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117851+0000", 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04171064 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117885+0000", 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3807999999999999e-05 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117913+0000", 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8229999999999999e-05 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260265 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463479548999999, 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042095026000000001, 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260265 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076145+0000", 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076147+0000", 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076152+0000", 2024-09-16T13:01:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5130000000000004e-06 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076156+0000", 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117924+0000", 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041768179000000002 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117980+0000", 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6125000000000002e-05 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118239+0000", 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025934200000000003 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260266 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.629 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463464681000001, 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045273031999999998, 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260266 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076160+0000", 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076161+0000", 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.796e-06 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076167+0000", 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-06 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076171+0000", 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.579e-06 2024-09-16T13:01:07.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.120966+0000", 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.044795297999999997 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121159+0000", 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192618 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121432+0000", 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000273663 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260267 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.463448304, 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046181535000000003, 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:07.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260267 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:07.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076178+0000", 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7969999999999999e-06 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076184+0000", 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6570000000000002e-06 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076187+0000", 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.455e-06 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122253+0000", 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046065813999999997 2024-09-16T13:01:07.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122318+0000", 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5338000000000004e-05 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122357+0000", 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9088999999999998e-05 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:07.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:07.639 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:07.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:07.642 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:01:07.643 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:01:07.644 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-16T13:01:06.752375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1146299903,"num_objects":2175,"num_object_clones":373,"num_object_copies":4350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2171,"num_whiteouts":1222,"num_read":328491,"num_read_kb":641662,"num_write":19470,"num_write_kb":309121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":32808300,"num_keys_recovered":159,"num_objects_omap":34,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16763,"ondisk_log_size":16763,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2471316,"kb_used_data":44748,"kb_used_omap":47,"kb_used_meta":2426192,"kb_avail":752503404,"statfs":{"total":773094113280,"available":770563485696,"internally_reserved":0,"allocated":45821952,"data_stored":309994349,"data_compressed":7325164,"data_compressed_allocated":20000768,"data_compressed_original":292733382,"omap_allocated":48222,"internal_metadata":2484421538},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-74806213,"num_objects":-188,"num_object_clones":-36,"num_object_copies":-376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-188,"num_whiteouts":-30,"num_read":1267,"num_read_kb":-742,"num_write":295,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":8792765,"num_keys_recovered":12,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.038210"},"pg_stats":[{"pgid":"4.1f","version":"118'286","reported_seq":4012,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.867485+0000","last_change":"2024-09-16T13:01:04.740119+0000","last_active":"2024-09-16T13:01:04.867485+0000","last_peered":"2024-09-16T13:01:04.867485+0000","last_clean":"2024-09-16T13:01:04.867485+0000","last_became_active":"2024-09-16T12:59:11.766798+0000","last_became_peered":"2024-09-16T12:59:11.766798+0000","last_unstale":"2024-09-16T13:01:04.867485+0000","last_undegraded":"2024-09-16T13:01:04.867485+0000","last_fullsized":"2024-09-16T13:01:04.867485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:22.588888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010856049999999999,"stat_sum":{"num_bytes":20230365,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":29,"num_read":3915,"num_read_kb":19318,"num_write":304,"num_write_kb":5430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.1e","version":"118'438","reported_seq":16880,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.889030+0000","last_change":"2024-09-16T13:01:04.733718+0000","last_active":"2024-09-16T13:01:04.889030+0000","last_peered":"2024-09-16T13:01:04.889030+0000","last_clean":"2024-09-16T13:01:04.889030+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:01:04.889030+0000","last_undegraded":"2024-09-16T13:01:04.889030+0000","last_fullsized":"2024-09-16T13:01:04.889030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010149479999999999,"stat_sum":{"num_bytes":39871549,"num_objects":83,"num_object_clones":17,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":45,"num_read":17148,"num_read_kb":19148,"num_write":438,"num_write_kb":9454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.1d","version":"118'395","reported_seq":4768,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.878002+0000","last_change":"2024-09-16T13:01:04.751965+0000","last_active":"2024-09-16T13:01:04.878002+0000","last_peered":"2024-09-16T13:01:04.878002+0000","last_clean":"2024-09-16T13:01:04.878002+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:01:04.878002+0000","last_undegraded":"2024-09-16T13:01:04.878002+0000","last_fullsized":"2024-09-16T13:01:04.878002+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001407291,"stat_sum":{"num_bytes":26066658,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":5128,"num_read_kb":15486,"num_write":334,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.1c","version":"118'1065","reported_seq":7514,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.995224+0000","last_change":"2024-09-16T13:01:04.732954+0000","last_active":"2024-09-16T13:01:04.995224+0000","last_peered":"2024-09-16T13:01:04.995224+0000","last_clean":"2024-09-16T13:01:04.995224+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:01:04.995224+0000","last_undegraded":"2024-09-16T13:01:04.995224+0000","last_fullsized":"2024-09-16T13:01:04.995224+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1065,"log_dups_size":0,"ondisk_log_size":1065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084944299999999996,"stat_sum":{"num_bytes":20416678,"num_objects":68,"num_object_clones":10,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":45,"num_read":9855,"num_read_kb":15477,"num_write":1424,"num_write_kb":5620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.1b","version":"116'386","reported_seq":21525,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.750098+0000","last_change":"2024-09-16T13:01:03.750906+0000","last_active":"2024-09-16T13:01:03.750098+0000","last_peered":"2024-09-16T13:01:03.750098+0000","last_clean":"2024-09-16T13:01:03.750098+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:01:03.750098+0000","last_undegraded":"2024-09-16T13:01:03.750098+0000","last_fullsized":"2024-09-16T13:01:03.750098+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:04.244300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004237228,"stat_sum":{"num_bytes":51579232,"num_objects":73,"num_object_clones":13,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":42,"num_read":22376,"num_read_kb":44613,"num_write":370,"num_write_kb":7677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"4.1a","version":"117'611","reported_seq":6653,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.917499+0000","last_change":"2024-09-16T13:01:03.747975+0000","last_active":"2024-09-16T13:01:03.917499+0000","last_peered":"2024-09-16T13:01:03.917499+0000","last_clean":"2024-09-16T13:01:03.917499+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:01:03.917499+0000","last_undegraded":"2024-09-16T13:01:03.917499+0000","last_fullsized":"2024-09-16T13:01:03.917499+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:30.964357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016225879999999999,"stat_sum":{"num_bytes":46793237,"num_objects":80,"num_object_clones":14,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":39,"num_read":7623,"num_read_kb":15684,"num_write":839,"num_write_kb":9750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"4.19","version":"118'353","reported_seq":5247,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.751392+0000","last_change":"2024-09-16T13:01:04.751392+0000","last_active":"2024-09-16T13:01:04.751392+0000","last_peered":"2024-09-16T13:01:04.751392+0000","last_clean":"2024-09-16T13:01:04.751392+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:01:04.751392+0000","last_undegraded":"2024-09-16T13:01:04.751392+0000","last_fullsized":"2024-09-16T13:01:04.751392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001144316,"stat_sum":{"num_bytes":20156471,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":36,"num_read":4887,"num_read_kb":18652,"num_write":322,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.18","version":"115'365","reported_seq":6135,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.746341+0000","last_change":"2024-09-16T13:01:03.746977+0000","last_active":"2024-09-16T13:01:03.746341+0000","last_peered":"2024-09-16T13:01:03.746341+0000","last_clean":"2024-09-16T13:01:03.746341+0000","last_became_active":"2024-09-16T13:00:58.691043+0000","last_became_peered":"2024-09-16T13:00:58.691043+0000","last_unstale":"2024-09-16T13:01:03.746341+0000","last_undegraded":"2024-09-16T13:01:03.746341+0000","last_fullsized":"2024-09-16T13:01:03.746341+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:32.993895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082299999999999995,"stat_sum":{"num_bytes":40794727,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":34,"num_read":6289,"num_read_kb":20336,"num_write":427,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"4.17","version":"118'787","reported_seq":7267,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.894313+0000","last_change":"2024-09-16T13:01:04.752637+0000","last_active":"2024-09-16T13:01:04.894313+0000","last_peered":"2024-09-16T13:01:04.894313+0000","last_clean":"2024-09-16T13:01:04.894313+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:01:04.894313+0000","last_undegraded":"2024-09-16T13:01:04.894313+0000","last_fullsized":"2024-09-16T13:01:04.894313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":787,"log_dups_size":0,"ondisk_log_size":787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012834999999999999,"stat_sum":{"num_bytes":63373085,"num_objects":78,"num_object_clones":12,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":42,"num_read":10635,"num_read_kb":28964,"num_write":974,"num_write_kb":16664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.16","version":"116'429","reported_seq":2873,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:05.022072+0000","last_change":"2024-09-16T13:01:04.733914+0000","last_active":"2024-09-16T13:01:05.022072+0000","last_peered":"2024-09-16T13:01:05.022072+0000","last_clean":"2024-09-16T13:01:05.022072+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:01:05.022072+0000","last_undegraded":"2024-09-16T13:01:05.022072+0000","last_fullsized":"2024-09-16T13:01:05.022072+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001024708,"stat_sum":{"num_bytes":25927058,"num_objects":70,"num_object_clones":10,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":44,"num_read":4637,"num_read_kb":11980,"num_write":376,"num_write_kb":2400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.15","version":"118'450","reported_seq":15883,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:05.051904+0000","last_change":"2024-09-16T13:01:04.734522+0000","last_active":"2024-09-16T13:01:05.051904+0000","last_peered":"2024-09-16T13:01:05.051904+0000","last_clean":"2024-09-16T13:01:05.051904+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:01:05.051904+0000","last_undegraded":"2024-09-16T13:01:05.051904+0000","last_fullsized":"2024-09-16T13:01:05.051904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081089400000000005,"stat_sum":{"num_bytes":37968482,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":40,"num_read":16219,"num_read_kb":17505,"num_write":513,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.14","version":"118'483","reported_seq":22862,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.888458+0000","last_change":"2024-09-16T13:01:04.752679+0000","last_active":"2024-09-16T13:01:04.888458+0000","last_peered":"2024-09-16T13:01:04.888458+0000","last_clean":"2024-09-16T13:01:04.888458+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:01:04.888458+0000","last_undegraded":"2024-09-16T13:01:04.888458+0000","last_fullsized":"2024-09-16T13:01:04.888458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017943639999999999,"stat_sum":{"num_bytes":66530080,"num_objects":74,"num_object_clones":18,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":39,"num_read":23272,"num_read_kb":34870,"num_write":532,"num_write_kb":16253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.13","version":"114'416","reported_seq":5721,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.747772+0000","last_change":"2024-09-16T13:01:03.748465+0000","last_active":"2024-09-16T13:01:03.747772+0000","last_peered":"2024-09-16T13:01:03.747772+0000","last_clean":"2024-09-16T13:01:03.747772+0000","last_became_active":"2024-09-16T13:01:00.705927+0000","last_became_peered":"2024-09-16T13:01:00.705927+0000","last_unstale":"2024-09-16T13:01:03.747772+0000","last_undegraded":"2024-09-16T13:01:03.747772+0000","last_fullsized":"2024-09-16T13:01:03.747772+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:31:26.553546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019540199999999999,"stat_sum":{"num_bytes":52792047,"num_objects":66,"num_object_clones":11,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":6225,"num_read_kb":27374,"num_write":425,"num_write_kb":42970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"4.12","version":"117'331","reported_seq":2899,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.733910+0000","last_change":"2024-09-16T13:01:04.734400+0000","last_active":"2024-09-16T13:01:04.733910+0000","last_peered":"2024-09-16T13:01:04.733910+0000","last_clean":"2024-09-16T13:01:04.733910+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:01:04.733910+0000","last_undegraded":"2024-09-16T13:01:04.733910+0000","last_fullsized":"2024-09-16T13:01:04.733910+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099173899999999995,"stat_sum":{"num_bytes":39159932,"num_objects":72,"num_object_clones":8,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":51,"num_read":2795,"num_read_kb":14285,"num_write":359,"num_write_kb":3309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.11","version":"118'338","reported_seq":18618,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.892457+0000","last_change":"2024-09-16T13:01:04.750670+0000","last_active":"2024-09-16T13:01:04.892457+0000","last_peered":"2024-09-16T13:01:04.892457+0000","last_clean":"2024-09-16T13:01:04.892457+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:01:04.892457+0000","last_undegraded":"2024-09-16T13:01:04.892457+0000","last_fullsized":"2024-09-16T13:01:04.892457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072246300000000001,"stat_sum":{"num_bytes":57990956,"num_objects":58,"num_object_clones":8,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":41,"num_read":18561,"num_read_kb":29302,"num_write":393,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.10","version":"118'305","reported_seq":5245,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.924421+0000","last_change":"2024-09-16T13:01:04.750610+0000","last_active":"2024-09-16T13:01:04.924421+0000","last_peered":"2024-09-16T13:01:04.924421+0000","last_clean":"2024-09-16T13:01:04.924421+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:01:04.924421+0000","last_undegraded":"2024-09-16T13:01:04.924421+0000","last_fullsized":"2024-09-16T13:01:04.924421+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071113899999999996,"stat_sum":{"num_bytes":14671801,"num_objects":68,"num_object_clones":10,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":48,"num_read":5067,"num_read_kb":16246,"num_write":308,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":108,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.845996+0000","last_change":"2024-09-16T12:58:56.032097+0000","last_active":"2024-09-16T13:01:03.845996+0000","last_peered":"2024-09-16T13:01:03.845996+0000","last_clean":"2024-09-16T13:01:03.845996+0000","last_became_active":"2024-09-16T12:58:53.636332+0000","last_became_peered":"2024-09-16T12:58:53.636332+0000","last_unstale":"2024-09-16T13:01:03.845996+0000","last_undegraded":"2024-09-16T13:01:03.845996+0000","last_fullsized":"2024-09-16T13:01:03.845996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:18.909469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098192,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"117'362","reported_seq":3276,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.922926+0000","last_change":"2024-09-16T13:01:03.747662+0000","last_active":"2024-09-16T13:01:03.922926+0000","last_peered":"2024-09-16T13:01:03.922926+0000","last_clean":"2024-09-16T13:01:03.922926+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:01:03.922926+0000","last_undegraded":"2024-09-16T13:01:03.922926+0000","last_fullsized":"2024-09-16T13:01:03.922926+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00130631,"stat_sum":{"num_bytes":29299156,"num_objects":54,"num_object_clones":5,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":33,"num_read":3953,"num_read_kb":3259,"num_write":303,"num_write_kb":3627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T13:01:02.736746+0000","last_change":"2024-09-16T12:59:40.993864+0000","last_active":"2024-09-16T13:01:02.736746+0000","last_peered":"2024-09-16T13:01:02.736746+0000","last_clean":"2024-09-16T13:01:02.736746+0000","last_became_active":"2024-09-16T12:59:40.992958+0000","last_became_peered":"2024-09-16T12:59:40.992958+0000","last_unstale":"2024-09-16T13:01:02.736746+0000","last_undegraded":"2024-09-16T13:01:02.736746+0000","last_fullsized":"2024-09-16T13:01:02.736746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:44.409493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"118'600","reported_seq":6861,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:05.003324+0000","last_change":"2024-09-16T13:01:04.751048+0000","last_active":"2024-09-16T13:01:05.003324+0000","last_peered":"2024-09-16T13:01:05.003324+0000","last_clean":"2024-09-16T13:01:05.003324+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:01:05.003324+0000","last_undegraded":"2024-09-16T13:01:05.003324+0000","last_fullsized":"2024-09-16T13:01:05.003324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085307899999999997,"stat_sum":{"num_bytes":29452645,"num_objects":86,"num_object_clones":16,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":54,"num_read":8707,"num_read_kb":16529,"num_write":550,"num_write_kb":6589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":109,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T13:01:02.717344+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:01:02.717344+0000","last_peered":"2024-09-16T13:01:02.717344+0000","last_clean":"2024-09-16T13:01:02.717344+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:01:02.717344+0000","last_undegraded":"2024-09-16T13:01:02.717344+0000","last_fullsized":"2024-09-16T13:01:02.717344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"118'521","reported_seq":2677,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.919440+0000","last_change":"2024-09-16T13:01:04.733666+0000","last_active":"2024-09-16T13:01:04.919440+0000","last_peered":"2024-09-16T13:01:04.919440+0000","last_clean":"2024-09-16T13:01:04.919440+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:01:04.919440+0000","last_undegraded":"2024-09-16T13:01:04.919440+0000","last_fullsized":"2024-09-16T13:01:04.919440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012957929999999999,"stat_sum":{"num_bytes":4765198,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":26,"num_read":2875,"num_read_kb":5451,"num_write":464,"num_write_kb":6178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T13:01:02.735761+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:01:02.735761+0000","last_peered":"2024-09-16T13:01:02.735761+0000","last_clean":"2024-09-16T13:01:02.735761+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:01:02.735761+0000","last_undegraded":"2024-09-16T13:01:02.735761+0000","last_fullsized":"2024-09-16T13:01:02.735761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"116'495","reported_seq":16684,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.733098+0000","last_change":"2024-09-16T13:01:03.733456+0000","last_active":"2024-09-16T13:01:03.733098+0000","last_peered":"2024-09-16T13:01:03.733098+0000","last_clean":"2024-09-16T13:01:03.733098+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:01:03.733098+0000","last_undegraded":"2024-09-16T13:01:03.733098+0000","last_fullsized":"2024-09-16T13:01:03.733098+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052060599999999996,"stat_sum":{"num_bytes":21675815,"num_objects":70,"num_object_clones":9,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":41,"num_read":17779,"num_read_kb":19876,"num_write":601,"num_write_kb":8612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.3","version":"0'0","reported_seq":71,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T13:01:02.747825+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:01:02.747825+0000","last_peered":"2024-09-16T13:01:02.747825+0000","last_clean":"2024-09-16T13:01:02.747825+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:01:02.747825+0000","last_undegraded":"2024-09-16T13:01:02.747825+0000","last_fullsized":"2024-09-16T13:01:02.747825+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":137,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.728242+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:01:03.728242+0000","last_peered":"2024-09-16T13:01:03.728242+0000","last_clean":"2024-09-16T13:01:03.728242+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:01:03.728242+0000","last_undegraded":"2024-09-16T13:01:03.728242+0000","last_fullsized":"2024-09-16T13:01:03.728242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"118'412","reported_seq":17710,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.894862+0000","last_change":"2024-09-16T13:01:04.739739+0000","last_active":"2024-09-16T13:01:04.894862+0000","last_peered":"2024-09-16T13:01:04.894862+0000","last_clean":"2024-09-16T13:01:04.894862+0000","last_became_active":"2024-09-16T12:59:56.105790+0000","last_became_peered":"2024-09-16T12:59:56.105790+0000","last_unstale":"2024-09-16T13:01:04.894862+0000","last_undegraded":"2024-09-16T13:01:04.894862+0000","last_fullsized":"2024-09-16T13:01:04.894862+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:32.594440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094010100000000004,"stat_sum":{"num_bytes":19584009,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":34,"num_read":19018,"num_read_kb":22665,"num_write":449,"num_write_kb":10611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"3.0","version":"0'0","reported_seq":68,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T13:01:02.735410+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:01:02.735410+0000","last_peered":"2024-09-16T13:01:02.735410+0000","last_clean":"2024-09-16T13:01:02.735410+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:01:02.735410+0000","last_undegraded":"2024-09-16T13:01:02.735410+0000","last_fullsized":"2024-09-16T13:01:02.735410+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":112,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.722657+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:01:03.722657+0000","last_peered":"2024-09-16T13:01:03.722657+0000","last_clean":"2024-09-16T13:01:03.722657+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:01:03.722657+0000","last_undegraded":"2024-09-16T13:01:03.722657+0000","last_fullsized":"2024-09-16T13:01:03.722657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"113'338","reported_seq":2516,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.875697+0000","last_change":"2024-09-16T13:01:03.733903+0000","last_active":"2024-09-16T13:01:03.875697+0000","last_peered":"2024-09-16T13:01:03.875697+0000","last_clean":"2024-09-16T13:01:03.875697+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:01:03.875697+0000","last_undegraded":"2024-09-16T13:01:03.875697+0000","last_fullsized":"2024-09-16T13:01:03.875697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079545199999999999,"stat_sum":{"num_bytes":55052986,"num_objects":65,"num_object_clones":14,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":30,"num_read":2775,"num_read_kb":19404,"num_write":382,"num_write_kb":9400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.1","version":"55'1","reported_seq":115,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.764142+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:01:03.764142+0000","last_peered":"2024-09-16T13:01:03.764142+0000","last_clean":"2024-09-16T13:01:03.764142+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:01:03.764142+0000","last_undegraded":"2024-09-16T13:01:03.764142+0000","last_fullsized":"2024-09-16T13:01:03.764142+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":127,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T13:01:02.717559+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:01:02.717559+0000","last_peered":"2024-09-16T13:01:02.717559+0000","last_clean":"2024-09-16T13:01:02.717559+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:01:02.717559+0000","last_undegraded":"2024-09-16T13:01:02.717559+0000","last_fullsized":"2024-09-16T13:01:02.717559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"118'354","reported_seq":6988,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:05.056908+0000","last_change":"2024-09-16T13:01:04.734038+0000","last_active":"2024-09-16T13:01:05.056908+0000","last_peered":"2024-09-16T13:01:05.056908+0000","last_clean":"2024-09-16T13:01:05.056908+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:01:05.056908+0000","last_undegraded":"2024-09-16T13:01:05.056908+0000","last_fullsized":"2024-09-16T13:01:05.056908+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097335300000000004,"stat_sum":{"num_bytes":32520618,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":34,"num_read":6877,"num_read_kb":14067,"num_write":291,"num_write_kb":7389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"3.2","version":"107'3","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.725205+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:01:03.725205+0000","last_peered":"2024-09-16T13:01:03.725205+0000","last_clean":"2024-09-16T13:01:03.725205+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:01:03.725205+0000","last_undegraded":"2024-09-16T13:01:03.725205+0000","last_fullsized":"2024-09-16T13:01:03.725205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":110,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.727800+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:01:03.727800+0000","last_peered":"2024-09-16T13:01:03.727800+0000","last_clean":"2024-09-16T13:01:03.727800+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:01:03.727800+0000","last_undegraded":"2024-09-16T13:01:03.727800+0000","last_fullsized":"2024-09-16T13:01:03.727800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":197,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T13:01:02.717422+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:01:02.717422+0000","last_peered":"2024-09-16T13:01:02.717422+0000","last_clean":"2024-09-16T13:01:02.717422+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:01:02.717422+0000","last_undegraded":"2024-09-16T13:01:02.717422+0000","last_fullsized":"2024-09-16T13:01:02.717422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"118'372","reported_seq":3483,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.989911+0000","last_change":"2024-09-16T13:01:04.739337+0000","last_active":"2024-09-16T13:01:04.989911+0000","last_peered":"2024-09-16T13:01:04.989911+0000","last_clean":"2024-09-16T13:01:04.989911+0000","last_became_active":"2024-09-16T13:01:00.706577+0000","last_became_peered":"2024-09-16T13:01:00.706577+0000","last_unstale":"2024-09-16T13:01:04.989911+0000","last_undegraded":"2024-09-16T13:01:04.989911+0000","last_fullsized":"2024-09-16T13:01:04.989911+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:45:39.448521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060330500000000005,"stat_sum":{"num_bytes":13075609,"num_objects":81,"num_object_clones":15,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":45,"num_read":3405,"num_read_kb":12818,"num_write":341,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.8","version":"118'892","reported_seq":18386,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:05.017765+0000","last_change":"2024-09-16T13:01:04.734240+0000","last_active":"2024-09-16T13:01:05.017765+0000","last_peered":"2024-09-16T13:01:05.017765+0000","last_clean":"2024-09-16T13:01:05.017765+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:01:05.017765+0000","last_undegraded":"2024-09-16T13:01:05.017765+0000","last_fullsized":"2024-09-16T13:01:05.017765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082237,"stat_sum":{"num_bytes":60832720,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":34,"num_read":20908,"num_read_kb":26883,"num_write":1238,"num_write_kb":13855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.9","version":"113'287","reported_seq":615,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.746664+0000","last_change":"2024-09-16T13:01:03.747184+0000","last_active":"2024-09-16T13:01:03.746664+0000","last_peered":"2024-09-16T13:01:03.746664+0000","last_clean":"2024-09-16T13:01:03.746664+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:01:03.746664+0000","last_undegraded":"2024-09-16T13:01:03.746664+0000","last_fullsized":"2024-09-16T13:01:03.746664+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088537999999999996,"stat_sum":{"num_bytes":13745800,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":27,"num_read":2910,"num_read_kb":6193,"num_write":280,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"4.a","version":"118'1263","reported_seq":6688,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:05.056754+0000","last_change":"2024-09-16T13:01:04.741427+0000","last_active":"2024-09-16T13:01:05.056754+0000","last_peered":"2024-09-16T13:01:05.056754+0000","last_clean":"2024-09-16T13:01:05.056754+0000","last_became_active":"2024-09-16T12:59:44.028122+0000","last_became_peered":"2024-09-16T12:59:44.028122+0000","last_unstale":"2024-09-16T13:01:05.056754+0000","last_undegraded":"2024-09-16T13:01:05.056754+0000","last_fullsized":"2024-09-16T13:01:05.056754+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:27:27.517524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027625219999999999,"stat_sum":{"num_bytes":46907783,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":34,"num_read":11262,"num_read_kb":20455,"num_write":1961,"num_write_kb":5295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":33178,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.b","version":"118'884","reported_seq":6346,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T13:01:04.733451+0000","last_change":"2024-09-16T13:01:04.733451+0000","last_active":"2024-09-16T13:01:04.733451+0000","last_peered":"2024-09-16T13:01:04.733451+0000","last_clean":"2024-09-16T13:01:04.733451+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:01:04.733451+0000","last_undegraded":"2024-09-16T13:01:04.733451+0000","last_fullsized":"2024-09-16T13:01:04.733451+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012598430000000001,"stat_sum":{"num_bytes":22651494,"num_objects":75,"num_object_clones":10,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":39,"num_read":8650,"num_read_kb":19953,"num_write":1336,"num_write_kb":7202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"7"}]},{"pgid":"4.c","version":"116'481","reported_seq":5016,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.748121+0000","last_change":"2024-09-16T13:01:03.749281+0000","last_active":"2024-09-16T13:01:03.748121+0000","last_peered":"2024-09-16T13:01:03.748121+0000","last_clean":"2024-09-16T13:01:03.748121+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:01:03.748121+0000","last_undegraded":"2024-09-16T13:01:03.748121+0000","last_fullsized":"2024-09-16T13:01:03.748121+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002663852,"stat_sum":{"num_bytes":38200999,"num_objects":66,"num_object_clones":12,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":39,"num_read":5617,"num_read_kb":15994,"num_write":472,"num_write_kb":7000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"4.d","version":"117'383","reported_seq":4090,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.751459+0000","last_change":"2024-09-16T13:01:03.751837+0000","last_active":"2024-09-16T13:01:03.751459+0000","last_peered":"2024-09-16T13:01:03.751459+0000","last_clean":"2024-09-16T13:01:03.751459+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:01:03.751459+0000","last_undegraded":"2024-09-16T13:01:03.751459+0000","last_fullsized":"2024-09-16T13:01:03.751459+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0052446719999999997,"stat_sum":{"num_bytes":41020710,"num_objects":59,"num_object_clones":7,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":36,"num_read":4972,"num_read_kb":14988,"num_write":384,"num_write_kb":2463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"4.e","version":"116'774","reported_seq":3636,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.749478+0000","last_change":"2024-09-16T13:01:03.750026+0000","last_active":"2024-09-16T13:01:03.749478+0000","last_peered":"2024-09-16T13:01:03.749478+0000","last_clean":"2024-09-16T13:01:03.749478+0000","last_became_active":"2024-09-16T13:01:00.705814+0000","last_became_peered":"2024-09-16T13:01:00.705814+0000","last_unstale":"2024-09-16T13:01:03.749478+0000","last_undegraded":"2024-09-16T13:01:03.749478+0000","last_fullsized":"2024-09-16T13:01:03.749478+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:20:25.838820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034071769999999999,"stat_sum":{"num_bytes":39185305,"num_objects":71,"num_object_clones":14,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":36,"num_read":5161,"num_read_kb":7234,"num_write":932,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"4.f","version":"115'828","reported_seq":36389,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T13:01:03.733041+0000","last_change":"2024-09-16T13:01:03.733346+0000","last_active":"2024-09-16T13:01:03.733041+0000","last_peered":"2024-09-16T13:01:03.733041+0000","last_clean":"2024-09-16T13:01:03.733041+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:01:03.733041+0000","last_undegraded":"2024-09-16T13:01:03.733041+0000","last_fullsized":"2024-09-16T13:01:03.733041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":828,"log_dups_size":0,"ondisk_log_size":828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041476000000000002,"stat_sum":{"num_bytes":53416228,"num_objects":87,"num_object_clones":16,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":44,"num_read":38924,"num_read_kb":66598,"num_write":1044,"num_write_kb":23492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"24","length":"6"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1145709433,"num_objects":2167,"num_object_clones":373,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":1219,"num_read":328425,"num_read_kb":641607,"num_write":19366,"num_write_kb":307490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":954,"num_bytes_recovered":32808262,"num_keys_recovered":159,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33951744,"data_stored":301252206,"data_compressed":7234443,"data_compressed_allocated":19771392,"data_compressed_original":289853894,"omap_allocated":9108,"internal_metadata":0},"log_size":16684,"ondisk_log_size":16684,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303124,"kb_used_data":5764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068716,"statfs":{"total":96636764160,"available":96326365184,"internally_reserved":0,"allocated":5902336,"data_stored":44787189,"data_compressed":1060231,"data_compressed_allocated":2609152,"data_compressed_original":42525962,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.876,"15min":0.90800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.131,"5min":1.393,"15min":1.393},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.88700000000000001,"15min":0.91400000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.755}]},{"osd":1,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.90100000000000002,"15min":0.92300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.216,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.024,"15min":0.97599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":8.2279999999999998,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":1.0529999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.90600000000000003,"15min":0.93600000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3300000000000001,"5min":1.518,"15min":1.518},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92200000000000004,"15min":0.94099999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.78600000000000003}]},{"osd":3,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.0069999999999999,"15min":0.96299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.1379999999999999,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":7.2069999999999999,"5min":7.2069999999999999,"15min":7.2069999999999999},"last":1.0049999999999999}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.82099999999999995}]},{"osd":5,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.98099999999999998,"15min":0.996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.419,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.131,"15min":1.173},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2869999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.98699999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":1.1799999999999999,"15min":1.24},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2230000000000001,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.1819999999999999,"15min":1.216},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4399999999999999,"5min":1.726,"15min":1.726},"last":1.196}]}]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301772,"kb_used_data":4416,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070068,"statfs":{"total":96636764160,"available":96327749632,"internally_reserved":0,"allocated":4521984,"data_stored":26503034,"data_compressed":527534,"data_compressed_allocated":1765376,"data_compressed_original":24682078,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.224,"5min":1.272,"15min":1.272},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2130000000000001,"5min":1.391,"15min":1.391},"last":0.76400000000000001}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":0.96899999999999997,"15min":0.89100000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.275,"5min":14.275,"15min":14.275},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.89800000000000002}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.999,"5min":1.095,"15min":1.111},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":1.129}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.097,"15min":1.075},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.872},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.0429999999999999}]},{"osd":4,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.80500000000000005}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":0.99099999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.1160000000000001,"15min":1.1499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4399999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.224,"15min":1.258},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.53,"5min":1.748,"15min":1.748},"last":1.2050000000000001}]}]},{"osd":4,"up_from":108,"seq":463856467971,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309764,"kb_used_data":6516,"kb_used_omap":35,"kb_used_meta":303196,"kb_avail":94062076,"statfs":{"total":96636764160,"available":96319565824,"internally_reserved":0,"allocated":6672384,"data_stored":50381030,"data_compressed":1164080,"data_compressed_allocated":2957312,"data_compressed_original":47829490,"omap_allocated":36294,"internal_metadata":310473274},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318348,"kb_used_data":8340,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94053492,"statfs":{"total":96636764160,"available":96310775808,"internally_reserved":0,"allocated":8540160,"data_stored":68446847,"data_compressed":1511591,"data_compressed_allocated":4386816,"data_compressed_original":65473236,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.33900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.70899999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76900000000000002,"15min":0.77800000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.94499999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.82599999999999996,"15min":0.80500000000000005},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.61099999999999999}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.75},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":0.98299999999999998}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1319999999999999,"15min":1.1359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.097,"15min":1.115},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.0669999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.113,"15min":1.1279999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.208,"15min":1.1919999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.3089999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509472,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313280,"kb_used_data":3444,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058560,"statfs":{"total":96636764160,"available":96315965440,"internally_reserved":0,"allocated":3526656,"data_stored":20955479,"data_compressed":630381,"data_compressed_allocated":1355776,"data_compressed_original":19592234,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.80000000000000004,"15min":0.75900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.654,"5min":10.654,"15min":10.654},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.244,"5min":0.84599999999999997,"15min":0.78000000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.60399999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.88400000000000001,"15min":0.86199999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":0.90700000000000003,"15min":0.80900000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":18.117000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.88300000000000001,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.75}]},{"osd":4,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.145},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.1220000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.159,"15min":1.2150000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.415,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.878,"5min":1.1180000000000001,"15min":1.1579999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5509999999999999,"5min":2.125,"15min":2.125},"last":0.89600000000000002}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.96899999999999997}]},{"osd":7,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.2010000000000001,"15min":1.2490000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.792,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.2010000000000001,"15min":1.2609999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3899999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.2290000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509473,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300656,"kb_used_data":3296,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071184,"statfs":{"total":96636764160,"available":96328892416,"internally_reserved":0,"allocated":3375104,"data_stored":14238855,"data_compressed":495719,"data_compressed_allocated":999424,"data_compressed_original":12840406,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":0.91300000000000003,"15min":0.85099999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.407,"5min":9.407,"15min":9.407},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.91100000000000003,"15min":0.96299999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.99399999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.123,"5min":2.254,"15min":2.254},"last":0.53800000000000003}]},{"osd":3,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.80100000000000005,"15min":0.79300000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.3460000000000001,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.997,"15min":1.0429999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.256,"5min":2.044,"15min":2.044},"last":0.73099999999999998}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.997},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.95499999999999996}]},{"osd":5,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.0449999999999999,"15min":1.0740000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4370000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.1819999999999999,"15min":1.222},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.617,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.85599999999999998}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.88}]},{"osd":7,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.163,"15min":1.1879999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6779999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.1319999999999999,"15min":1.167},"min":{"1min":0.438,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.583,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.93600000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542176,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302508,"kb_used_data":5152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069332,"statfs":{"total":96636764160,"available":96326995968,"internally_reserved":0,"allocated":5275648,"data_stored":32253094,"data_compressed":748966,"data_compressed_allocated":2265088,"data_compressed_original":30310730,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.2370000000000001,"15min":1.2909999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7470000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.86699999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.125,"15min":1.1890000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.681,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.115,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.071,"15min":1.069},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.2110000000000001,"15min":1.2250000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.089}]},{"osd":3,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0449999999999999,"15min":1.083},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2869999999999999,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.159},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.2649999999999999,"15min":1.319},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.623,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":0.90200000000000002}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.7889999999999997,"5min":4.7889999999999997,"15min":4.7889999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.91600000000000004}]},{"osd":5,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.284,"15min":1.339},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.75,"5min":6.282,"15min":6.282},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.96799999999999997,"15min":0.98799999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.611,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.879}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.61799999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321864,"kb_used_data":7820,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94049976,"statfs":{"total":96636764160,"available":96307175424,"internally_reserved":0,"allocated":8007680,"data_stored":52428821,"data_compressed":1186662,"data_compressed_allocated":3661824,"data_compressed_original":49479246,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85099999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":1.3380000000000001}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76600000000000001,"15min":0.78900000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.234,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.86199999999999999,"15min":0.88800000000000001},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.077,"5min":3.024,"15min":3.024},"last":1.181}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":1.004,"15min":1.048},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.103,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.97699999999999998}]},{"osd":4,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.90600000000000003}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.349,"15min":1.431},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.234,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.071,"15min":1.0880000000000001},"min":{"1min":0.59799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":1.139}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.294},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.238}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1299999999999999,"15min":1.147},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.381},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.27,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.2669999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1912832,"data_stored":12955480,"data_compressed":486381,"data_compressed_allocated":962560,"data_compressed_original":12266966,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6422528,"data_stored":51399870,"data_compressed":1179517,"data_compressed_allocated":3641344,"data_compressed_original":49266254,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2129920,"data_stored":20263012,"data_compressed":630381,"data_compressed_allocated":1355776,"data_compressed_original":19592234,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7135232,"data_stored":67753804,"data_compressed":1511591,"data_compressed_allocated":4386816,"data_compressed_original":65473236,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5083136,"data_stored":49346097,"data_compressed":1160058,"data_compressed_allocated":2940928,"data_compressed_original":47620594,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2936832,"data_stored":24463286,"data_compressed":466656,"data_compressed_allocated":1646592,"data_compressed_original":23371358,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4501504,"data_stored":44094229,"data_compressed":1060231,"data_compressed_allocated":2609152,"data_compressed_original":42525962,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3829760,"data_stored":30976428,"data_compressed":739628,"data_compressed_allocated":2228224,"data_compressed_original":29737290,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:01:07.647 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T13:01:07.647 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T13:01:07.648 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-16T13:01:07.648 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T13:01:07.648 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:01:07.648 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:07.637+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:07.717 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T13:01:07.717 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2024-09-16T13:01:07.718 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T13:01:07.718 DEBUG:teuthology.orchestra.run.smithi136:> type ceph-objectstore-tool 2024-09-16T13:01:07.723 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T13:01:07.723 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-09-16T13:01:07.740 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:07.732+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:07.753 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:07.753 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:07.763 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T13:01:07.766 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:07.841 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:07.836+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:07.904 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:01:07.918 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:07.918 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:07.918 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:07.918 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:07.918 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:07.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:07.969 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:07.964+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:08.031 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:08.042 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:08.037+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:08.057 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:08.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T13:01:08.057+0000 7fbdb7a7d300 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T13:01:08.075 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:08.075 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:08.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:08.075 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:08.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.075 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:08.075 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:08.075 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.002507903000001, 2024-09-16T13:01:08.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:01:08.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:01:08.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:01:08.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:01:08.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:01:08.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:01:08.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:01:08.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:01:08.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:01:08.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:01:08.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:01:08.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:01:08.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:01:08.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.082 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.083 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:08.083 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:08.083 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.002455484000002, 2024-09-16T13:01:08.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:01:08.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:01:08.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.084 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:08.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:08.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:08.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:01:08.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:01:08.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:01:08.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.089 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.089 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:08.089 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:08.089 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.002438636000001, 2024-09-16T13:01:08.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:01:08.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.092 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:01:08.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.092 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:08.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.002422522000003, 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:01:08.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:01:08.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.002357592000003, 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:01:08.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:08.105 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.002341493000003, 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:08.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:01:08.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:01:08.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:01:08.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:01:08.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:01:08.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.002267711999998, 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:08.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:01:08.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:01:08.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:01:08.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:08.115 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.002250511, 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:08.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:01:08.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.002233808, 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:01:08.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.465922552999999, 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:01:08.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:01:08.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:01:08.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.465905417000002, 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:08.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:01:08.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:08.133 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.465885835000002, 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:08.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.465854976000003, 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:01:08.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.465786880000003, 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:08.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:01:08.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.458483025, 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:08.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:08.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:08.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:01:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.458465435999997, 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:01:08.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:01:08.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.458450542999998, 2024-09-16T13:01:08.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:01:08.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.160 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.458418926, 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:01:08.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:01:08.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:08.165 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.458400904999998, 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:01:08.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.544365533000001, 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2569454119999999, 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 279128 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527093+0000", 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4040000000000002e-06 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527104+0000", 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-05 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527133+0000", 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8909999999999999e-05 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527144+0000", 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1497e-05 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527223+0000", 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8577000000000003e-05 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527236+0000", 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3362e-05 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783387+0000", 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2561508619999999 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783699+0000", 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311687 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.784036+0000", 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033771300000000001 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:08.175 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:08.176 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:08.177 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:08.178 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:08.178 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:08.178 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:08.178 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:08.179 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:08.179 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:08.179 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:08.179 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:08.179 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:08.179 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:08.180 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:08.180 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:08.180 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:08.180 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:08.180 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:08.168+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:08.278 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:08.273+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:08.286 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:08.286 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:08.296 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:08.379 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:08.372+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:08.398 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:08.398 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:08.407 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:08.480 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:08.477+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:08.506 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:08.506 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:08.516 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:08.582 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:08.576+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:08.615 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:08.615 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:08.625 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:08.683 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:08.676+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T13:01:08.734 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:08.734 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:08.743 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:08.783 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:08.781+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:08.836 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:08.836 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:08.845 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:08.885 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:08.876+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:08.942 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:08.942 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:08.951 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:08.986 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:08.981+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:09.055 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:09.055 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:09.064 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:09.111 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:09.109+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:09.187 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:09.185+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:09.209 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:09.209 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:09.218 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:09.288 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:09.280+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:09.329 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:09.329 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:09.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-16T13:01:09.329+0000 7fbdb7a7d300 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T13:01:09.339 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:09.389 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:09.384+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:09.489 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:09.489 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:09.489 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:09.484+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:09.498 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:09.590 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:09.584+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:09.649 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:09.649 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:09.658 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:09.691 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:09.689+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:09.760 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:09.760 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:09.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:09.793 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:09.789+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:09.880 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:09.880 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:09.890 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:09.893 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:09.884+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:09.899 INFO:teuthology.orchestra.run.smithi136.stdout:4.e 2024-09-16T13:01:09.899 INFO:teuthology.orchestra.run.smithi136.stdout:4.a 2024-09-16T13:01:09.899 INFO:teuthology.orchestra.run.smithi136.stdout:4.1f 2024-09-16T13:01:09.899 INFO:teuthology.orchestra.run.smithi136.stdout:4.5 2024-09-16T13:01:09.899 INFO:teuthology.orchestra.run.smithi136.stdout:4.4 2024-09-16T13:01:09.899 INFO:teuthology.orchestra.run.smithi136.stdout:4.13 2024-09-16T13:01:09.899 INFO:teuthology.orchestra.run.smithi136.stdout:4.18 2024-09-16T13:01:09.899 INFO:teuthology.orchestra.run.smithi136.stdout:2.7 2024-09-16T13:01:09.899 INFO:teuthology.orchestra.run.smithi136.stdout:2.6 2024-09-16T13:01:09.991 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:09.992 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:09.994 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:09.989+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:10.001 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:10.095 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:10.088+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:10.100 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:10.101 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:10.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:10.182 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:01:10.196 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:10.193+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:10.210 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:10.210 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:10.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:10.297 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:10.293+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:10.325 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:10.331 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:10.331 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:10.345 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:10.353 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:10.386 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:10.398 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:10.392+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:10.409 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:10.430 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:10.450 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 4.18 --file /var/log/ceph/exp.4.18.6 2024-09-16T13:01:10.451 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:10.451 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:10.451 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:10.454 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:10.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:10.478 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:10.478 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:10.478 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:10.478 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:10.478 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:10.478 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:10.478 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:10.479 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:10.479 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:10.481 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:10.482 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:10.499 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:10.492+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:10.552 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:10.552 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:10.552 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:10.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:10.605 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:10.600+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:10.701 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:10.696+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:10.711 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:10.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.637461958999999, 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:01:10.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.716 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.63740954, 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:10.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:01:10.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.637392691999999, 2024-09-16T13:01:10.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:01:10.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.637376578000001, 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:01:10.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:10.730 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.637311648000001, 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:10.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:01:10.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.637295549000001, 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:10.739 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.637221768000003, 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:01:10.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.637204566999998, 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.744 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:01:10.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.637187863999998, 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:01:10.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:01:10.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:01:10.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.100876608999997, 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:01:10.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:01:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:01:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:01:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:01:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.100859473, 2024-09-16T13:01:10.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:10.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.100839891, 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.764 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:10.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.100809032000001, 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:10.772 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.100740936000001, 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:01:10.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:01:10.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.093437080999998, 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:01:10.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:01:10.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.093419492000002, 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.093404599000003, 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:01:10.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:01:10.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.093372981999998, 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.791 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:01:10.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.093354961000003, 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:01:10.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:01:10.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.179319589, 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2569454119999999, 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 279128 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:10.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527093+0000", 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4040000000000002e-06 2024-09-16T13:01:10.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527104+0000", 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-05 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527133+0000", 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8909999999999999e-05 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527144+0000", 2024-09-16T13:01:10.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1497e-05 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527223+0000", 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8577000000000003e-05 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527236+0000", 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3362e-05 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:10.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783387+0000", 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2561508619999999 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783699+0000", 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311687 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.784036+0000", 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033771300000000001 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:10.805 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:10.805 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:10.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T13:01:10.807 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:10.807 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:10.800+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:10.908 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:10.904+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:10.916 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:10.916 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:10.916 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:10.916 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:10.927 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:11.009 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:11.004+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:11.028 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:11.029 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:11.029 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:11.037 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:11.110 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:11.108+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:11.135 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:11.136 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:11.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:11.136 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:11.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.136 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240664 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 152576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.137 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:11.137 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.587703317999999, 2024-09-16T13:01:11.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047790817999999999, 2024-09-16T13:01:11.137 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.137 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.137 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.137 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.138 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.138 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240664 2024-09-16T13:01:11.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.138 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:11.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:11.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:11.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1900000000000003e-07 2024-09-16T13:01:11.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543064+0000", 2024-09-16T13:01:11.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4449999999999999e-06 2024-09-16T13:01:11.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543079+0000", 2024-09-16T13:01:11.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5221e-05 2024-09-16T13:01:11.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543084+0000", 2024-09-16T13:01:11.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8590000000000004e-06 2024-09-16T13:01:11.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590750+0000", 2024-09-16T13:01:11.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047665710999999999 2024-09-16T13:01:11.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590801+0000", 2024-09-16T13:01:11.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1827999999999999e-05 2024-09-16T13:01:11.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590852+0000", 2024-09-16T13:01:11.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0235000000000003e-05 2024-09-16T13:01:11.145 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.145 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.145 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.145 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240665 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.145 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:11.145 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.587673915, 2024-09-16T13:01:11.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047896192999999997, 2024-09-16T13:01:11.146 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.146 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.146 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240665 2024-09-16T13:01:11.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.147 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:11.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:11.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543091+0000", 2024-09-16T13:01:11.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543094+0000", 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7149999999999998e-06 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543102+0000", 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6549999999999993e-06 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543108+0000", 2024-09-16T13:01:11.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4840000000000003e-06 2024-09-16T13:01:11.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590885+0000", 2024-09-16T13:01:11.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047777396 2024-09-16T13:01:11.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590936+0000", 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0611e-05 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590987+0000", 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0785000000000003e-05 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.151 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240666 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 162816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.587651647000001, 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048010069000000002, 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240666 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8500000000000002e-07 2024-09-16T13:01:11.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543116+0000", 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.531e-06 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543123+0000", 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1069999999999999e-06 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543127+0000", 2024-09-16T13:01:11.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4309999999999996e-06 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591020+0000", 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047892413000000002 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591071+0000", 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1650000000000002e-05 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591123+0000", 2024-09-16T13:01:11.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1452000000000001e-05 2024-09-16T13:01:11.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.156 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.156 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240667 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.587632865, 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048125951, 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240667 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543132+0000", 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5099999999999999e-07 2024-09-16T13:01:11.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543134+0000", 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1500000000000002e-06 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543141+0000", 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543146+0000", 2024-09-16T13:01:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9200000000000003e-06 2024-09-16T13:01:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591158+0000", 2024-09-16T13:01:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048011246 2024-09-16T13:01:11.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591207+0000", 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9295999999999997e-05 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591257+0000", 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0426000000000001e-05 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240668 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 173056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.58761316, 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048243280999999999, 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240668 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.162 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543152+0000", 2024-09-16T13:01:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543154+0000", 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5380000000000001e-06 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543161+0000", 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7920000000000004e-06 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543172+0000", 2024-09-16T13:01:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1327e-05 2024-09-16T13:01:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591290+0000", 2024-09-16T13:01:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048118181000000003 2024-09-16T13:01:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591344+0000", 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4061000000000003e-05 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591394+0000", 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9866e-05 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.166 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240766 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.586408835, 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047716644000000003, 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240766 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544356+0000", 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2500000000000005e-07 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544373+0000", 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6685999999999999e-05 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544381+0000", 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7190000000000001e-06 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544384+0000", 2024-09-16T13:01:11.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591969+0000", 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047585483999999997 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592021+0000", 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1839000000000003e-05 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592072+0000", 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754000000000002e-05 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240767 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 201728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.586376959999999, 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047822479000000001, 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.171 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240767 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544388+0000", 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.51e-07 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544390+0000", 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.824e-06 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544393+0000", 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.681e-06 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544396+0000", 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4940000000000002e-06 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592105+0000", 2024-09-16T13:01:11.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047709184000000002 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592158+0000", 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2772999999999998e-05 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592210+0000", 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2071999999999999e-05 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240768 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.586364158999999, 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047944647, 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240768 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.176 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:11.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0400000000000002e-07 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544403+0000", 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1430000000000001e-06 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544406+0000", 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7450000000000001e-06 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544409+0000", 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592241+0000", 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047832146999999998 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592291+0000", 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9289000000000002e-05 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592345+0000", 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4067000000000003e-05 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240769 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.586352534, 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048066225999999997, 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240769 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:11.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1399999999999998e-07 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544413+0000", 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.42e-06 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544417+0000", 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.529e-06 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544420+0000", 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5620000000000002e-06 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592378+0000", 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047958354000000002 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592427+0000", 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9013e-05 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592478+0000", 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1034e-05 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240770 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:11.184 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.586342270999999, 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048189432999999997, 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240770 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:11.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3200000000000001e-07 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544424+0000", 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4449999999999999e-06 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544427+0000", 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5049999999999999e-06 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544430+0000", 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.61e-06 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592511+0000", 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048080912000000003 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592563+0000", 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2107000000000001e-05 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592611+0000", 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8522000000000001e-05 2024-09-16T13:01:11.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240771 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.586307664, 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290128000000002, 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.189 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240771 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5700000000000002e-07 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544459+0000", 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6840000000000001e-06 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544466+0000", 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999997e-06 2024-09-16T13:01:11.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544469+0000", 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592645+0000", 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048175216999999999 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592696+0000", 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1292999999999998e-05 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592747+0000", 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0751000000000002e-05 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240772 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.193 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.586284369000001, 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048403333999999999, 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240772 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:11.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544482+0000", 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8390000000000001e-06 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544486+0000", 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9369999999999997e-06 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544489+0000", 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8459999999999999e-06 2024-09-16T13:01:11.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592779+0000", 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290301000000001 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592831+0000", 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2335e-05 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592883+0000", 2024-09-16T13:01:11.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1786999999999997e-05 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240773 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.586272984000001, 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048527963, 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.198 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240773 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544492+0000", 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.84e-07 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544493+0000", 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6279999999999999e-06 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544498+0000", 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4689999999999999e-06 2024-09-16T13:01:11.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544500+0000", 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592916+0000", 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048415672 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592965+0000", 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9120999999999999e-05 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593019+0000", 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3950999999999999e-05 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240774 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:11.202 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.586261328999999, 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048649546000000002, 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240774 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:11.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:01:11.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544505+0000", 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.61e-06 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544508+0000", 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4929999999999998e-06 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544511+0000", 2024-09-16T13:01:11.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4169999999999999e-06 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593051+0000", 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048539604 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593103+0000", 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2068999999999999e-05 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593152+0000", 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9870999999999997e-05 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240775 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.586251075, 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048773328999999997, 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240775 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545618+0000", 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011045810000000001 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545625+0000", 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6079999999999998e-06 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545644+0000", 2024-09-16T13:01:11.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.823e-05 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545653+0000", 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3000000000000007e-06 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593186+0000", 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047533387000000003 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593235+0000", 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8767000000000003e-05 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593286+0000", 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1456000000000003e-05 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240776 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.585098937000001, 2024-09-16T13:01:11.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047754889000000002, 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240776 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545666+0000", 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-06 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545671+0000", 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2270000000000004e-06 2024-09-16T13:01:11.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545681+0000", 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0506000000000001e-05 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545687+0000", 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0970000000000002e-06 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593320+0000", 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047632999000000002 2024-09-16T13:01:11.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593369+0000", 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8882999999999999e-05 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593420+0000", 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1127e-05 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.215 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240777 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.585070315999999, 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047878318000000003, 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240777 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:11.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545695+0000", 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545698+0000", 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0709999999999999e-06 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545707+0000", 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7629999999999995e-06 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545713+0000", 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2839999999999997e-06 2024-09-16T13:01:11.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593453+0000", 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047740281000000002 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593518+0000", 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5493000000000005e-05 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593572+0000", 2024-09-16T13:01:11.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3739000000000002e-05 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240778 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.585045876999999, 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047999411999999998, 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240778 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:11.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545719+0000", 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.13e-07 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545728+0000", 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1560000000000001e-06 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545901+0000", 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017311599999999999 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545909+0000", 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.028e-06 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593606+0000", 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047696660000000002 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593666+0000", 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9611999999999998e-05 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593718+0000", 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2227e-05 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.224 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240951 4.8 4:10cc0d5f:::rbd_data.11b552946c0f.0000000000000120:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.576965607999998, 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048226894999999999, 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240951 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2300000000000002e-07 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553801+0000", 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.686e-06 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553805+0000", 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3039999999999998e-06 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553809+0000", 2024-09-16T13:01:11.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.599994+0000", 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046185154999999999 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.600112+0000", 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118589 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.602026+0000", 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019131370000000001 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:272626 4.12 4:4935e950:::rbd_data.11b59088eb24.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.672937986000001, 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022439990000001, 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.229 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 272626 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457828+0000", 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457835+0000", 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7900000000000002e-06 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457853+0000", 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7671000000000001e-05 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457859+0000", 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2949999999999999e-06 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457913+0000", 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4112e-05 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457922+0000", 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0850000000000003e-06 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.457879+0000", 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.99995641700000004 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458079+0000", 2024-09-16T13:01:11.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020072300000000001 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458304+0000", 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000224601 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459393+0000", 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010886170000000001 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459877+0000", 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048443500000000002 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460006+0000", 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012914 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460070+0000", 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4074999999999999e-05 2024-09-16T13:01:11.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.236 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.236 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.236 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:11.237 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:11.238 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:11.232+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:11.338 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:11.336+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:11.346 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:11.346 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:11.346 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:11.346 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:11.355 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:11.439 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:11.436+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:11.503 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:11.503 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:11.503 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:11.503 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:11.503 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:11.512 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:11.540 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:11.536+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:11.615 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 146.06089324800001, 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:01:11.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:11.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:01:11.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:01:11.622 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.23577620099999, 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.623 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:01:11.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.628 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.23572894500001, 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:01:11.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:01:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.23552600900001, 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:01:11.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:01:11.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.23549474699999, 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:01:11.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:01:11.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:01:11.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:01:11.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:01:11.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.23545898500001, 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.647 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.648 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.648 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:01:11.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.648 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:11.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:11.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:01:11.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:01:11.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:01:11.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:01:11.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:01:11.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:01:11.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:01:11.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.235264355, 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.655 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:11.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:01:11.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:01:11.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.23521751000001, 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.661 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.662 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:01:11.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.662 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:01:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:01:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.69094133900001, 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:01:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:01:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.69088743200001, 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:01:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:01:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.69084512699999, 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:01:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:01:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:01:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.69068670199999, 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:01:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:01:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.690637221, 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:01:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:01:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.69050665099999, 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:01:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:01:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.69032513799999, 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:01:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:01:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 143.87979747099999, 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:01:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 143.87964851699999, 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:01:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:01:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:01:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64507 4.1e e83/31 4:7f7c7834:::rbd_data.11b5c2ee3542.0000000000000008:head v 83'225, mlcod=81'223)", 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.840235229999998, 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021284401000000001, 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770380+0000", 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8999999999999998e-07 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777141+0000", 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067617670000000001 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777157+0000", 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777167+0000", 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777267+0000", 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100364 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777304+0000", 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7165000000000001e-05 2024-09-16T13:01:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791601+0000", 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014297275999999999 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791664+0000", 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2217000000000004e-05 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64509 4.15 e83/31 4:ad2e18f4:::rbd_data.11b5c2ee3542.0000000000000009:head v 83'183, mlcod=81'180)", 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.833477328000001, 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014054395000000001, 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777138+0000", 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778534+0000", 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013956769999999999 2024-09-16T13:01:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778555+0000", 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1046000000000001e-05 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778564+0000", 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9290000000000004e-06 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778633+0000", 2024-09-16T13:01:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9363000000000007e-05 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778666+0000", 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135000000000003e-05 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791145+0000", 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012479743 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791192+0000", 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6156000000000003e-05 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:217754 4.17 e92/31 4:ee4048fa:::rbd_data.11b58362ce83.0000000000000002:head v 92'697, mlcod=91'693)", 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.111307273999998, 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011416822, 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499309+0000", 2024-09-16T13:01:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6929999999999999e-06 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499322+0000", 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499354+0000", 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135999999999999e-05 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499365+0000", 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111e-05 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499504+0000", 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013877599999999999 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499565+0000", 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1939000000000004e-05 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510591+0000", 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011025874999999999 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510724+0000", 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013266299999999999 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:11.735 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:11.737 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:11.743 INFO:teuthology.orchestra.run.smithi136.stderr:Exporting 4.18 info 4.18( v 120'378 (0'0,120'378] local-lis/les=111/112 n=63 ec=31/31 lis/c=111/111 les/c/f=112/112/0 sis=111) 2024-09-16T13:01:11.743 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:18176585:::rbd_data.11b552946c0f.0000000000000153:head# 2024-09-16T13:01:11.743 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:182b09dc:::rbd_data.11b552946c0f.000000000000011e:head# 2024-09-16T13:01:11.743 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:183aa6f6:::rbd_data.11b5a5cd9d85.0000000000000077:head# 2024-09-16T13:01:11.743 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:18688110:::rbd_data.11b5b635ee69.00000000000000da:17# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:18688110:::rbd_data.11b5b635ee69.00000000000000da:head# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:18991311:::rbd_data.11b5b3dc7343.00000000000000bf:head# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:18b02557:::rbd_data.11b54010928d.00000000000000aa:1b# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:18b02557:::rbd_data.11b54010928d.00000000000000aa:head# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:18b6860e:::rbd_data.11b5403f243a.000000000000018a:head# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:18ca1bd6:::rbd_data.11b5a5cd9d85.0000000000000013:head# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:18f89e6a:::rbd_data.11b571860f2a.0000000000000001:11# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:18f89e6a:::rbd_data.11b571860f2a.0000000000000001:head# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:193bbcf1:::rbd_id.image_client.0-clone20:head# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1940ff59:::rbd_data.11b5a5cd9d85.0000000000000000:head# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:19649cbc:::rbd_data.11b5403f243a.0000000000000112:head# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:19682d6f:::rbd_data.11b552946c0f.0000000000000156:head# 2024-09-16T13:01:11.744 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1969d45c:::rbd_data.11b5403f243a.0000000000000086:head# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:196cb0e9:::rbd_data.11b552946c0f.0000000000000318:head# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:19828ede:::rbd_data.11b58f77e1c4.0000000000000015:head# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:19c668f7:::rbd_data.11b515823229.00000000000000ec:head# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:19cffc52:::rbd_data.11b5b1b9bb41.000000000000000f:head# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:19e0f116:::rbd_data.11b515823229.00000000000000c7:1e# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:19e0f116:::rbd_data.11b515823229.00000000000000c7:head# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1a178a80:::rbd_data.11b5a5cd9d85.0000000000000044:head# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1a33e3b5:::rbd_data.11b552946c0f.00000000000002b7:head# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1a62f4ff:::rbd_data.11b5403f243a.000000000000006d:head# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1a8060ab:::rbd_data.11b5403f243a.0000000000000146:head# 2024-09-16T13:01:11.745 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1a87f04a:::rbd_data.11b552946c0f.0000000000000296:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1a8b08a8:::rbd_data.11b54010928d.0000000000000138:1b# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1a8b08a8:::rbd_data.11b54010928d.0000000000000138:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1a93aa1d:::rbd_data.11b5403f243a.000000000000011b:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1b367f8f:::rbd_data.11b552946c0f.0000000000000182:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1b3e433f:::rbd_data.11b515823229.00000000000000fd:1e# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1b3e433f:::rbd_data.11b515823229.00000000000000fd:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1b518b96:::rbd_data.11b5403f243a.0000000000000082:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1b55a29f:::rbd_data.11b5b635ee69.0000000000000164:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1bb5c193:::rbd_data.11b552946c0f.0000000000000268:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1bba35a0:::rbd_data.11b5403f243a.0000000000000081:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1c9f5a62:::rbd_data.11b515823229.00000000000000ff:1e# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1c9f5a62:::rbd_data.11b515823229.00000000000000ff:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1cb1d743:::rbd_data.11b552946c0f.0000000000000300:head# 2024-09-16T13:01:11.746 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1d022c83:::rbd_data.11b5403f243a.0000000000000131:head# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1d4e9176:::rbd_data.11b5b635ee69.0000000000000002:17# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1d4e9176:::rbd_data.11b5b635ee69.0000000000000002:head# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1d706ff5:::rbd_data.11b5403f243a.000000000000014a:head# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1d85c3a1:::rbd_data.11b5b635ee69.0000000000000006:17# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1d85c3a1:::rbd_data.11b5b635ee69.0000000000000006:head# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1d89336a:::rbd_object_map.11b5a5cd9d85:head# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1d8ce1a8:::rbd_data.11b552946c0f.000000000000012d:head# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1d920efd:::rbd_data.11b5b635ee69.00000000000000e5:17# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1d920efd:::rbd_data.11b5b635ee69.00000000000000e5:head# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1dce9f1e:::rbd_data.11b5a5cd9d85.0000000000000007:head# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1ded3ca7:::rbd_data.11b59d02cc38.0000000000000055:head# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1e21ada4:::rbd_object_map.11b5c2ee3542.000000000000001d:head# 2024-09-16T13:01:11.747 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1ea2ddcc:::rbd_data.11b5403f243a.0000000000000116:head# 2024-09-16T13:01:11.748 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1edc95aa:::rbd_data.11b54010928d.0000000000000109:1b# 2024-09-16T13:01:11.748 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1edc95aa:::rbd_data.11b54010928d.0000000000000109:head# 2024-09-16T13:01:11.748 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1ef20a43:::rbd_data.11b552946c0f.0000000000000154:head# 2024-09-16T13:01:11.748 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1f49b46c:::rbd_object_map.11b53140b009:head# 2024-09-16T13:01:11.748 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1f580d39:::rbd_data.11b552946c0f.000000000000025b:head# 2024-09-16T13:01:11.748 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1f629f1e:::rbd_data.11b5403f243a.00000000000000fe:head# 2024-09-16T13:01:11.748 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1f717d35:::rbd_data.11b5403f243a.0000000000000073:head# 2024-09-16T13:01:11.748 INFO:teuthology.orchestra.run.smithi136.stderr:Read #4:1fdd37ab:::rbd_data.11b5403f243a.000000000000008e:head# 2024-09-16T13:01:11.748 INFO:teuthology.orchestra.run.smithi136.stderr:Export successful 2024-09-16T13:01:11.749 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:11.732+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:11.808 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T13:01:11.858 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:11.858 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:11.868 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:11.869 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:11.864+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:11.938 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:11.932+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:11.973 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:11.974 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:11.974 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:11.974 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:11.974 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:11.983 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:12.039 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:12.036+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:12.082 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.544570646, 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98766548899999995, 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279128 2024-09-16T13:01:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534648+0000", 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1459999999999999e-06 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534655+0000", 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5220000000000004e-06 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534674+0000", 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9717000000000001e-05 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534683+0000", 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534759+0000", 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6074000000000004e-05 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534770+0000", 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0665999999999999e-05 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522281+0000", 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98751079600000002 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522312+0000", 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0876000000000001e-05 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.9 109/108 [PushOp(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29, version: 106'278, data_included: [0~688128,802816~10077,884736~295,1950271~32193], data_size: 730693, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29@106'278, size: 1982464, copy_subset: [0~1982464], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1982464, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.437551514000001, 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067875690000000002, 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641666+0000", 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3060000000000001e-06 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644856+0000", 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00318995 2024-09-16T13:01:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644894+0000", 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7382999999999998e-05 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644913+0000", 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9415999999999998e-05 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645084+0000", 2024-09-16T13:01:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017036899999999999 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645106+0000", 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2887e-05 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.648453+0000", 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003346258 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279793 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.744160+0000", 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3350563110000007, 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004380274, 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279793 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744160+0000", 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744160+0000", 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744163+0000", 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.067e-06 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744176+0000", 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2184999999999999e-05 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744207+0000", 2024-09-16T13:01:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1754000000000001e-05 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744219+0000", 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1457999999999999e-05 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744341+0000", 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122101 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.744351+0000", 2024-09-16T13:01:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0222e-05 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.748076+0000", 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037249679999999999 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.748277+0000", 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020039299999999999 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.748541+0000", 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000264126 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279882 4.c 4:3656acfa:::rbd_data.11b53ee6e6bb.0000000000000009:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.2105743499999999, 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004921407, 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279882 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868645+0000", 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5550000000000001e-06 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868656+0000", 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868691+0000", 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4459999999999999e-05 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868703+0000", 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2556e-05 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868818+0000", 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011502000000000001 2024-09-16T13:01:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.869209+0000", 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039085699999999999 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.869563+0000", 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035391699999999998 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.871027+0000", 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00146393 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873450+0000", 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002422914 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873527+0000", 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7286000000000005e-05 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873564+0000", 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6553000000000001e-05 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280527 4.13 4:cc227b3c:::rbd_data.11b5cb8797c6.0000000000000000:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.2060921569999996, 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052047300000000003, 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280527 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873126+0000", 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6530000000000001e-06 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873133+0000", 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0940000000000002e-06 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873153+0000", 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9538000000000002e-05 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873161+0000", 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1009999999999992e-06 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873245+0000", 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4075999999999999e-05 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873360+0000", 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114951 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873530+0000", 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016994699999999999 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.877617+0000", 2024-09-16T13:01:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040873000000000003 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878160+0000", 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054299199999999995 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878251+0000", 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1062000000000003e-05 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878329+0000", 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8016000000000007e-05 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280530 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.2060301530000004, 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056839330000000004, 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280530 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8199999999999999e-07 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873191+0000", 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4599999999999999e-06 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873198+0000", 2024-09-16T13:01:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8399999999999997e-06 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873201+0000", 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.641e-06 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873298+0000", 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6682999999999996e-05 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873451+0000", 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015304000000000001 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873692+0000", 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024124300000000001 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.876380+0000", 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026873980000000001 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878771+0000", 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002391013 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878824+0000", 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3136000000000001e-05 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878870+0000", 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6796999999999999e-05 2024-09-16T13:01:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280624 4.1b 4:d8560506:::rbd_header.11b5de89dd5b:head [watch watch cookie 140717155157888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.999647+0000", 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.079569845, 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004107388, 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280624 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999647+0000", 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999647+0000", 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999649+0000", 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.48e-06 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999663+0000", 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3623e-05 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999694+0000", 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0729999999999999e-05 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999707+0000", 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3152999999999999e-05 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.999847+0000", 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140605 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.000097+0000", 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024959100000000002 2024-09-16T13:01:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.000584+0000", 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048667799999999999 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.002009+0000", 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001424795 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.003370+0000", 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001361686 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.003520+0000", 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014938199999999999 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.003754+0000", 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000234665 2024-09-16T13:01:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280650 4.1b 4:d8560506:::rbd_header.11b5de89dd5b:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3197057430000001, 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085055370000000005, 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280650 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759514+0000", 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7429999999999998e-06 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759524+0000", 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8309999999999996e-06 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759540+0000", 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6506000000000001e-05 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759548+0000", 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1810000000000001e-06 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.761995+0000", 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002446518 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.762120+0000", 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125124 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.762945+0000", 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082531599999999998 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.766013+0000", 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030673549999999999 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.767864+0000", 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001851161 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.767939+0000", 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5012000000000006e-05 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.768016+0000", 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7789999999999999e-05 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280740 4.18 4:1df486ae:::rbd_data.11b5de89dd5b.0000000000000000:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1702136139999997, 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070806940000000002, 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280740 2024-09-16T13:01:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909005+0000", 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.317e-06 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909016+0000", 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0852999999999999e-05 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909052+0000", 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5256000000000003e-05 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909064+0000", 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2413e-05 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909168+0000", 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103615 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909335+0000", 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016715100000000001 2024-09-16T13:01:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909599+0000", 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000264391 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.914767+0000", 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051679309999999997 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915839+0000", 2024-09-16T13:01:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001072415 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915986+0000", 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014609099999999999 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.916084+0000", 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8261e-05 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280741 4.13 4:ce26f199:::rbd_data.11b5de89dd5b.0000000000000001:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1701270260000003, 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062230289999999997, 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280741 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909091+0000", 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0100000000000001e-06 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909096+0000", 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4399999999999996e-06 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909107+0000", 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.066e-05 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909113+0000", 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9460000000000003e-06 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909206+0000", 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3450999999999997e-05 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909341+0000", 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135267 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909519+0000", 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177603 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.913993+0000", 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044734989999999997 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915108+0000", 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001115001 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915207+0000", 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9562999999999995e-05 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915313+0000", 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010558899999999999 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281211 4.5 4:a14f106d:::rbd_header.11b515823229:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2815800130000001, 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093462500000000004, 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281211 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797641+0000", 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4259999999999996e-06 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797662+0000", 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.069e-05 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797705+0000", 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3495999999999998e-05 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797719+0000", 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4003e-05 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797828+0000", 2024-09-16T13:01:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010857 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797840+0000", 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.235e-05 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.802951+0000", 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051111079999999996 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.803147+0000", 2024-09-16T13:01:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000196109 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806806+0000", 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003658246 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806865+0000", 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9246999999999997e-05 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806983+0000", 2024-09-16T13:01:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011800500000000001 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281301 4.9 4:97313ac6:::rbd_data.11b515823229.0000000000000021:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.924415+0000", 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1548014580000001, 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039120609999999997, 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281301 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924415+0000", 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924415+0000", 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924418+0000", 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3690000000000001e-06 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924427+0000", 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4050000000000006e-06 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924454+0000", 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6597e-05 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924464+0000", 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0319000000000001e-05 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924563+0000", 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9443000000000003e-05 2024-09-16T13:01:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924718+0000", 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015421900000000001 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924918+0000", 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020012200000000001 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.927387+0000", 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024692049999999999 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928243+0000", 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085623000000000003 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928287+0000", 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3477999999999999e-05 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928327+0000", 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0673999999999997e-05 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281307 4.c 4:32a25aa2:::rbd_data.11b515823229.000000000000002b:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.924498+0000", 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.15471838, 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048053389999999996, 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281307 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924498+0000", 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924498+0000", 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924499+0000", 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8299999999999997e-07 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924502+0000", 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924510+0000", 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6320000000000008e-06 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924515+0000", 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4710000000000001e-06 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924648+0000", 2024-09-16T13:01:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013367300000000001 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924794+0000", 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014601600000000001 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924905+0000", 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111082 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.927599+0000", 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026937440000000001 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929141+0000", 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015415069999999999 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929229+0000", 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7967000000000007e-05 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929304+0000", 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5094999999999998e-05 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281308 4.1b 4:db2b1562:::rbd_data.11b515823229.000000000000002f:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.154681002, 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056950619999999999, 2024-09-16T13:01:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281308 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7999999999999995e-07 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924539+0000", 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9799999999999998e-06 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924546+0000", 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4620000000000001e-06 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924550+0000", 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0040000000000001e-06 2024-09-16T13:01:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924628+0000", 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7749000000000004e-05 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924853+0000", 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000225491 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.925042+0000", 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018871099999999999 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928196+0000", 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031538120000000002 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930092+0000", 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001896556 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930167+0000", 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5111999999999995e-05 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930231+0000", 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3504999999999996e-05 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281311 4.1 4:86a487b7:::rbd_data.11b515823229.000000000000007a:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.928494+0000", 2024-09-16T13:01:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1507224329999999, 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043931079999999997, 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281311 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928494+0000", 2024-09-16T13:01:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928494+0000", 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928496+0000", 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.72e-06 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928501+0000", 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8389999999999997e-06 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928514+0000", 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3475999999999999e-05 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928521+0000", 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.297e-06 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928567+0000", 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6078000000000002e-05 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928684+0000", 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117472 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928787+0000", 2024-09-16T13:01:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010288 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930433+0000", 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001645786 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.932776+0000", 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023433769999999998 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.932832+0000", 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5631999999999999e-05 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.932887+0000", 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5550999999999997e-05 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281312 4.9 4:9026e48e:::rbd_data.11b515823229.000000000000007b:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.928536+0000", 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.150681106, 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004227583, 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281312 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928536+0000", 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928536+0000", 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928536+0000", 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1099999999999996e-07 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928539+0000", 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5950000000000001e-06 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928545+0000", 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3720000000000001e-06 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928549+0000", 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9230000000000004e-06 2024-09-16T13:01:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928593+0000", 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3705000000000003e-05 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928682+0000", 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9438000000000007e-05 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928779+0000", 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6798000000000004e-05 2024-09-16T13:01:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.932257+0000", 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034780660000000001 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.932586+0000", 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032949999999999999 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.932687+0000", 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100302 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.932763+0000", 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6372999999999995e-05 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281315 4.e 4:7397209c:::rbd_data.11b515823229.000000000000007e:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.928566+0000", 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1506503509999999, 2024-09-16T13:01:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042767860000000003, 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281315 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928566+0000", 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928566+0000", 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928567+0000", 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0100000000000005e-07 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928569+0000", 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3599999999999999e-06 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928575+0000", 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0859999999999998e-06 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928580+0000", 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2710000000000003e-06 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928633+0000", 2024-09-16T13:01:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2907e-05 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928758+0000", 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012585499999999999 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.932679+0000", 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039208070000000001 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.932754+0000", 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5032999999999998e-05 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.932843+0000", 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8865999999999999e-05 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281318 4.e 4:7158d438:::rbd_data.11b515823229.0000000000000081:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1497951020000001, 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058318850000000002, 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281318 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929423+0000", 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2139999999999999e-06 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929428+0000", 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.826e-06 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929439+0000", 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1549000000000001e-05 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929444+0000", 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0050000000000004e-06 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929506+0000", 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1927999999999994e-05 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929632+0000", 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012569299999999999 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935090+0000", 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054580920000000003 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935154+0000", 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3688000000000004e-05 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935253+0000", 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9889999999999994e-05 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281319 4.1b 4:d932f5f4:::rbd_data.11b515823229.0000000000000082:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.930789+0000", 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.148427238, 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044561430000000001, 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281319 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930789+0000", 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930789+0000", 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930791+0000", 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2270000000000001e-06 2024-09-16T13:01:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930797+0000", 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9519999999999999e-06 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930813+0000", 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6164999999999999e-05 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930820+0000", 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.289e-06 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930873+0000", 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2984e-05 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.931016+0000", 2024-09-16T13:01:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142507 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.931134+0000", 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011879699999999999 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.933761+0000", 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026264249999999999 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935133+0000", 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013722300000000001 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935195+0000", 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1925000000000001e-05 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935246+0000", 2024-09-16T13:01:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0642000000000001e-05 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281500 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:head [delete] snapc 1d=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:11.928207+0000", 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.15101003900000001, 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047239270000000002, 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281500 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928207+0000", 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928207+0000", 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928208+0000", 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6139999999999999e-06 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928217+0000", 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5569999999999996e-06 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928239+0000", 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2357e-05 2024-09-16T13:01:12.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928249+0000", 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8079999999999999e-06 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928339+0000", 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0224999999999998e-05 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928499+0000", 2024-09-16T13:01:12.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015945600000000001 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928838+0000", 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000339792 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.930649+0000", 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001810063 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:12.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.932806+0000", 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021571379999999999 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.932879+0000", 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3543999999999999e-05 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.932931+0000", 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1372999999999998e-05 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.196 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.197 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.197 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:12.197 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:12.197 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:12.198 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:12.199 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:12.192+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:12.263 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 4.18 2024-09-16T13:01:12.299 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:12.292+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:12.308 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:12.309 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:12.309 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:12.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:12.400 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:12.396+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:12.420 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:12.420 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:12.420 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:12.429 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:12.483 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:01:12.501 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:12.496+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:12.537 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:12.537 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.839016364999999, 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:01:12.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:01:12.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.839001338999999, 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:12.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838985317000002, 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:01:12.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:01:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838969220999999, 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:01:12.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838953330999999, 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:01:12.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:01:12.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:12.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838937661000003, 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838921913999997, 2024-09-16T13:01:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:01:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:01:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:01:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:01:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838906125999998, 2024-09-16T13:01:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:01:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:01:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:01:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838890804000002, 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838870878999998, 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:01:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:01:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838854931, 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:01:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.83883977, 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838824121999998, 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:01:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:01:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:01:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838809025000003, 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:01:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:01:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838793762999998, 2024-09-16T13:01:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:01:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:01:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838778017000003, 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:01:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838762420000002, 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:01:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838747116, 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:01:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:01:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838731678000002, 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:01:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 58.838715964999999, 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:01:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:01:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:01:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:01:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:12.632 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:12.634 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:12.634 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:12.635 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:12.628+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:12.639 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:12.671 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:12.705 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:12.726 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:12.734 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:12.728+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:12.741 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:12.741 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:12.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:12.752 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:12.772 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:12.772 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:12.772 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:12.772 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:12.772 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:12.772 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:12.772 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:12.772 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:12.773 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:12.775 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:12.776 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:12.835 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:12.828+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:12.852 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:12.852 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:12.862 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:12.936 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:12.928+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:13.007 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:13.007 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:13.017 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:13.037 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:13.032+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:13.112 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:13.112 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:13.122 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:13.165 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:13.160+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:13.239 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:13.236+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:13.268 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:13.268 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:13.278 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:13.341 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:13.332+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:13.378 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:13.378 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:13.387 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:13.414 INFO:teuthology.orchestra.run.smithi136.stdout: marking collection for removal 2024-09-16T13:01:13.414 INFO:teuthology.orchestra.run.smithi136.stdout:setting '_remove' omap key 2024-09-16T13:01:13.422 INFO:teuthology.orchestra.run.smithi136.stdout:finish_remove_pgs 4.18_head removing 4.18 2024-09-16T13:01:13.434 INFO:teuthology.orchestra.run.smithi136.stdout:Remove successful 2024-09-16T13:01:13.441 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:13.436+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:13.531 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:13.531 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:13.540 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:13.542 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:13.536+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:13.642 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:13.642 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:13.642 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:13.636+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:13.652 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:13.743 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:13.740+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:13.774 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:13.774 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:13.784 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:13.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T13:01:13.869 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:13.864+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:13.893 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:13.893 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:13.903 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:13.944 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:13.940+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:13.965 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.4.18.6 2024-09-16T13:01:14.003 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:14.003 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:14.012 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:14.045 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:14.040+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:14.102 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:14.102 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:14.111 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:14.155 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:14.152+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:14.247 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:14.240+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:14.256 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:14.256 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:14.265 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:14.348 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:14.344+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:14.369 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:14.369 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:14.378 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:14.449 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:14.444+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:14.525 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:14.526 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:14.534 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:14.550 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:14.544+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:14.633 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:14.633 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:14.642 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:14.650 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:14.644+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:14.738 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:14.739 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:14.747 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:14.751 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:14.748+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:14.776 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:01:14.824 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T13:01:14.845 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:14.845 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:14.851 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:14.848+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:14.854 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:14.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:14.952 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:14.948+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:14.958 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:14.983 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:14.990 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:14.990 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:14.990 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:15.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:15.008 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:15.045 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:15.053 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:15.048+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:15.071 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:15.105 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:15.105 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:15.105 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:15.106 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:15.106 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:15.106 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:15.106 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:15.106 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:15.106 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:15.109 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:15.109 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:15.111 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:15.111 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:15.111 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:15.111 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:15.111 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:15.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:15.141 INFO:teuthology.orchestra.run.smithi136.stdout:get_pg_num_history pg_num_history pg_num_history(e120 pg_nums {1={15=1},2={18=8},3={23=4},4={31=32}} deleted_pools ) 2024-09-16T13:01:15.141 INFO:teuthology.orchestra.run.smithi136.stdout:Importing pgid 4.18 2024-09-16T13:01:15.149 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:18176585:::rbd_data.11b552946c0f.0000000000000153:head# 2024-09-16T13:01:15.149 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.149 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:182b09dc:::rbd_data.11b552946c0f.000000000000011e:head# 2024-09-16T13:01:15.149 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.150 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:183aa6f6:::rbd_data.11b5a5cd9d85.0000000000000077:head# 2024-09-16T13:01:15.151 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.153 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:18688110:::rbd_data.11b5b635ee69.00000000000000da:17# 2024-09-16T13:01:15.154 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:18688110:::rbd_data.11b5b635ee69.00000000000000da:head# 2024-09-16T13:01:15.155 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 17=[]:{17=[17]} 2024-09-16T13:01:15.156 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:18991311:::rbd_data.11b5b3dc7343.00000000000000bf:head# 2024-09-16T13:01:15.156 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.157 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:18b02557:::rbd_data.11b54010928d.00000000000000aa:1b# 2024-09-16T13:01:15.158 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:18b02557:::rbd_data.11b54010928d.00000000000000aa:head# 2024-09-16T13:01:15.158 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1b=[]:{1b=[1b]} 2024-09-16T13:01:15.158 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:18b6860e:::rbd_data.11b5403f243a.000000000000018a:head# 2024-09-16T13:01:15.158 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.159 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:18ca1bd6:::rbd_data.11b5a5cd9d85.0000000000000013:head# 2024-09-16T13:01:15.159 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.160 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:18f89e6a:::rbd_data.11b571860f2a.0000000000000001:11# 2024-09-16T13:01:15.165 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:15.160+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:15.167 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:18f89e6a:::rbd_data.11b571860f2a.0000000000000001:head# 2024-09-16T13:01:15.169 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 11=[]:{11=[11]} 2024-09-16T13:01:15.172 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:193bbcf1:::rbd_id.image_client.0-clone20:head# 2024-09-16T13:01:15.172 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.173 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1940ff59:::rbd_data.11b5a5cd9d85.0000000000000000:head# 2024-09-16T13:01:15.174 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.176 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:19649cbc:::rbd_data.11b5403f243a.0000000000000112:head# 2024-09-16T13:01:15.176 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.176 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:19682d6f:::rbd_data.11b552946c0f.0000000000000156:head# 2024-09-16T13:01:15.176 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.177 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1969d45c:::rbd_data.11b5403f243a.0000000000000086:head# 2024-09-16T13:01:15.177 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.177 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:196cb0e9:::rbd_data.11b552946c0f.0000000000000318:head# 2024-09-16T13:01:15.177 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.178 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:19828ede:::rbd_data.11b58f77e1c4.0000000000000015:head# 2024-09-16T13:01:15.178 INFO:teuthology.orchestra.run.smithi136.stdout:snapset c=[]:{} 2024-09-16T13:01:15.178 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:19c668f7:::rbd_data.11b515823229.00000000000000ec:head# 2024-09-16T13:01:15.178 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1e=[]:{} 2024-09-16T13:01:15.179 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:19cffc52:::rbd_data.11b5b1b9bb41.000000000000000f:head# 2024-09-16T13:01:15.180 INFO:teuthology.orchestra.run.smithi136.stdout:snapset a=[]:{} 2024-09-16T13:01:15.183 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:19e0f116:::rbd_data.11b515823229.00000000000000c7:1e# 2024-09-16T13:01:15.185 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:19e0f116:::rbd_data.11b515823229.00000000000000c7:head# 2024-09-16T13:01:15.185 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1e=[]:{1e=[1e]} 2024-09-16T13:01:15.185 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1a178a80:::rbd_data.11b5a5cd9d85.0000000000000044:head# 2024-09-16T13:01:15.186 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.188 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1a33e3b5:::rbd_data.11b552946c0f.00000000000002b7:head# 2024-09-16T13:01:15.188 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.188 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1a62f4ff:::rbd_data.11b5403f243a.000000000000006d:head# 2024-09-16T13:01:15.188 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.189 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1a8060ab:::rbd_data.11b5403f243a.0000000000000146:head# 2024-09-16T13:01:15.189 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.189 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1a87f04a:::rbd_data.11b552946c0f.0000000000000296:head# 2024-09-16T13:01:15.189 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.189 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1a8b08a8:::rbd_data.11b54010928d.0000000000000138:1b# 2024-09-16T13:01:15.190 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1a8b08a8:::rbd_data.11b54010928d.0000000000000138:head# 2024-09-16T13:01:15.190 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1b=[]:{1b=[1b]} 2024-09-16T13:01:15.191 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1a93aa1d:::rbd_data.11b5403f243a.000000000000011b:head# 2024-09-16T13:01:15.191 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.191 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1b367f8f:::rbd_data.11b552946c0f.0000000000000182:head# 2024-09-16T13:01:15.192 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.192 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1b3e433f:::rbd_data.11b515823229.00000000000000fd:1e# 2024-09-16T13:01:15.194 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1b3e433f:::rbd_data.11b515823229.00000000000000fd:head# 2024-09-16T13:01:15.194 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1e=[]:{1e=[1e]} 2024-09-16T13:01:15.195 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1b518b96:::rbd_data.11b5403f243a.0000000000000082:head# 2024-09-16T13:01:15.195 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.195 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1b55a29f:::rbd_data.11b5b635ee69.0000000000000164:head# 2024-09-16T13:01:15.195 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.197 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1bb5c193:::rbd_data.11b552946c0f.0000000000000268:head# 2024-09-16T13:01:15.197 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.197 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1bba35a0:::rbd_data.11b5403f243a.0000000000000081:head# 2024-09-16T13:01:15.197 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.198 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1c9f5a62:::rbd_data.11b515823229.00000000000000ff:1e# 2024-09-16T13:01:15.199 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1c9f5a62:::rbd_data.11b515823229.00000000000000ff:head# 2024-09-16T13:01:15.199 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1e=[]:{1e=[1e]} 2024-09-16T13:01:15.200 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1cb1d743:::rbd_data.11b552946c0f.0000000000000300:head# 2024-09-16T13:01:15.200 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.200 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1d022c83:::rbd_data.11b5403f243a.0000000000000131:head# 2024-09-16T13:01:15.200 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.201 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1d4e9176:::rbd_data.11b5b635ee69.0000000000000002:17# 2024-09-16T13:01:15.203 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1d4e9176:::rbd_data.11b5b635ee69.0000000000000002:head# 2024-09-16T13:01:15.203 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 17=[]:{17=[17]} 2024-09-16T13:01:15.204 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1d706ff5:::rbd_data.11b5403f243a.000000000000014a:head# 2024-09-16T13:01:15.204 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.205 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1d85c3a1:::rbd_data.11b5b635ee69.0000000000000006:17# 2024-09-16T13:01:15.206 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1d85c3a1:::rbd_data.11b5b635ee69.0000000000000006:head# 2024-09-16T13:01:15.207 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 17=[]:{17=[17]} 2024-09-16T13:01:15.208 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1d89336a:::rbd_object_map.11b5a5cd9d85:head# 2024-09-16T13:01:15.208 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.209 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1d8ce1a8:::rbd_data.11b552946c0f.000000000000012d:head# 2024-09-16T13:01:15.209 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.209 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1d920efd:::rbd_data.11b5b635ee69.00000000000000e5:17# 2024-09-16T13:01:15.211 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1d920efd:::rbd_data.11b5b635ee69.00000000000000e5:head# 2024-09-16T13:01:15.211 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 17=[]:{17=[17]} 2024-09-16T13:01:15.212 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1dce9f1e:::rbd_data.11b5a5cd9d85.0000000000000007:head# 2024-09-16T13:01:15.213 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.216 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1ded3ca7:::rbd_data.11b59d02cc38.0000000000000055:head# 2024-09-16T13:01:15.216 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.218 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1e21ada4:::rbd_object_map.11b5c2ee3542.000000000000001d:head# 2024-09-16T13:01:15.218 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.219 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1ea2ddcc:::rbd_data.11b5403f243a.0000000000000116:head# 2024-09-16T13:01:15.219 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.220 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1edc95aa:::rbd_data.11b54010928d.0000000000000109:1b# 2024-09-16T13:01:15.221 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1edc95aa:::rbd_data.11b54010928d.0000000000000109:head# 2024-09-16T13:01:15.221 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1b=[]:{1b=[1b]} 2024-09-16T13:01:15.222 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1ef20a43:::rbd_data.11b552946c0f.0000000000000154:head# 2024-09-16T13:01:15.222 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.223 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1f49b46c:::rbd_object_map.11b53140b009:head# 2024-09-16T13:01:15.223 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 0=[]:{} 2024-09-16T13:01:15.224 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:15.224 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:15.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:15.224 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:15.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.224 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.224 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:15.224 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525790491999999, 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:15.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:01:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525775465999999, 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:01:15.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:01:15.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:01:15.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:01:15.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:01:15.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525759444000002, 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:15.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:01:15.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.238 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525743347999999, 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:01:15.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:15.243 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525727457999999, 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:15.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:15.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525711788000002, 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:01:15.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:01:15.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.252 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525696041000003, 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:01:15.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.254 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:15.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:15.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:15.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:01:15.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:01:15.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:01:15.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:01:15.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:01:15.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:01:15.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:01:15.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:01:15.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:01:15.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:01:15.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:01:15.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:01:15.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:01:15.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.261 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.261 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.261 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:15.262 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525680252999997, 2024-09-16T13:01:15.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:01:15.262 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.262 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.263 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:01:15.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.263 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:15.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:15.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:01:15.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:01:15.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:01:15.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:01:15.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:01:15.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:01:15.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:01:15.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:01:15.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:01:15.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:01:15.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:01:15.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:01:15.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525664931000001, 2024-09-16T13:01:15.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:01:15.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.270 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.270 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.270 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.270 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:01:15.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.271 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:15.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:01:15.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:01:15.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:01:15.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.275 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.275 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:15.276 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525645005999998, 2024-09-16T13:01:15.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:01:15.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:01:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:01:15.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:01:15.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:01:15.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:01:15.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:01:15.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525629058, 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:01:15.282 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:01:15.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:01:15.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525613896999999, 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:15.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:01:15.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:01:15.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:01:15.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:01:15.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:01:15.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:01:15.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:01:15.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:01:15.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:01:15.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:01:15.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.293 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.293 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.293 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:15.294 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525598248999998, 2024-09-16T13:01:15.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:01:15.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:01:15.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.295 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:15.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:15.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525583152000003, 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:01:15.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525567889999998, 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:01:15.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:01:15.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.308 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525552144000002, 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:01:15.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525536547000002, 2024-09-16T13:01:15.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525521243, 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.318 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525505805000002, 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.525490091999998, 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.327 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:15.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:15.333 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1f580d39:::rbd_data.11b552946c0f.000000000000025b:head# 2024-09-16T13:01:15.333 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 24=[]:{} 2024-09-16T13:01:15.333 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1f629f1e:::rbd_data.11b5403f243a.00000000000000fe:head# 2024-09-16T13:01:15.333 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.334 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1f717d35:::rbd_data.11b5403f243a.0000000000000073:head# 2024-09-16T13:01:15.334 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.334 INFO:teuthology.orchestra.run.smithi136.stdout:Write #4:1fdd37ab:::rbd_data.11b5403f243a.000000000000008e:head# 2024-09-16T13:01:15.334 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1f=[]:{} 2024-09-16T13:01:15.334 INFO:teuthology.orchestra.run.smithi136.stdout:write_pg epoch 120 info 4.18( v 120'378 (0'0,120'378] local-lis/les=111/112 n=63 ec=31/31 lis/c=111/111 les/c/f=112/112/0 sis=111) 2024-09-16T13:01:15.334 INFO:teuthology.orchestra.run.smithi136.stdout:Import successful 2024-09-16T13:01:15.334 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:15.335 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:15.324+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:15.434 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:15.428+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:15.438 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:15.438 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:15.438 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:15.448 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:15.535 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:15.528+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:15.547 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:15.547 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:15.547 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:15.547 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:15.547 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:15.556 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:15.636 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:15.632+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:15.660 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254136 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 136192~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.806101727000001, 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082717923999999998, 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254136 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849492+0000", 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849494+0000", 2024-09-16T13:01:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0540000000000002e-06 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849500+0000", 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2899999999999999e-06 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849504+0000", 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8120000000000001e-06 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.931987+0000", 2024-09-16T13:01:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082483231000000004 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932061+0000", 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4181e-05 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932209+0000", 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014796399999999999 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254137 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 139264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.806085452000001, 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082930443000000006, 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254137 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849510+0000", 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0530000000000001e-06 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849516+0000", 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8359999999999997e-06 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849520+0000", 2024-09-16T13:01:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932239+0000", 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082719191999999997 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932307+0000", 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7824000000000007e-05 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932438+0000", 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000131354 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254138 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 146432~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.806069522000001, 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083100513000000001, 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254138 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7300000000000001e-07 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849526+0000", 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9089999999999998e-06 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849532+0000", 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6420000000000004e-06 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849535+0000", 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7740000000000002e-06 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932467+0000", 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082931469999999993 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932535+0000", 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7815e-05 2024-09-16T13:01:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932624+0000", 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9430000000000003e-05 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254139 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 149504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.806053965, 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083267068, 2024-09-16T13:01:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254139 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849540+0000", 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849542+0000", 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9379999999999999e-06 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849547+0000", 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849551+0000", 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5880000000000002e-06 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932652+0000", 2024-09-16T13:01:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083101176999999998 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932719+0000", 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.703e-05 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932806+0000", 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7340000000000001e-05 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254140 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 156672~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.806038692000001, 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083436222000000004, 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254140 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849557+0000", 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8419999999999999e-06 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849564+0000", 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7079999999999997e-06 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849569+0000", 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.386e-06 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932833+0000", 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083264099999999994 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932900+0000", 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7007000000000004e-05 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932991+0000", 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0815000000000004e-05 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254141 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.806020140000001, 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083599532000000004, 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254141 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849574+0000", 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1699999999999998e-07 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849576+0000", 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849582+0000", 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9769999999999997e-06 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849585+0000", 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933018+0000", 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083432372000000005 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933086+0000", 2024-09-16T13:01:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7862999999999997e-05 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933173+0000", 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7050000000000002e-05 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254142 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.806003990000001, 2024-09-16T13:01:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083767603999999996, 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254142 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849590+0000", 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.51e-07 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849592+0000", 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0360000000000001e-06 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849598+0000", 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4139999999999999e-06 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849602+0000", 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.659e-06 2024-09-16T13:01:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933199+0000", 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083597150999999995 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933267+0000", 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7933000000000002e-05 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933357+0000", 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9859999999999997e-05 2024-09-16T13:01:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254143 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.805987042000002, 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083932426000000004, 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254143 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849607+0000", 2024-09-16T13:01:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849609+0000", 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.165e-06 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849615+0000", 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2029999999999997e-06 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849619+0000", 2024-09-16T13:01:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933382+0000", 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083763447000000005 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933447+0000", 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4993000000000006e-05 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933539+0000", 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1434000000000006e-05 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254145 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.805970479999999, 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084130242999999993, 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254145 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851596+0000", 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019732550000000001 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851602+0000", 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5720000000000002e-06 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851610+0000", 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5529999999999993e-06 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851616+0000", 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6409999999999999e-06 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933580+0000", 2024-09-16T13:01:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.081963854000000003 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933653+0000", 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3901999999999998e-05 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933753+0000", 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9465999999999999e-05 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254149 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.803953957000001, 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082661199000000005, 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254149 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851640+0000", 2024-09-16T13:01:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851642+0000", 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851648+0000", 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9200000000000001e-06 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851652+0000", 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0439999999999997e-06 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934105+0000", 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082453495000000002 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934193+0000", 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7433000000000001e-05 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934300+0000", 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107513 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254151 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.803932335999999, 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082910496, 2024-09-16T13:01:15.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254151 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1799999999999995e-07 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851664+0000", 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4399999999999999e-06 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851670+0000", 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3509999999999998e-06 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851674+0000", 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.912e-06 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934385+0000", 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082710396000000005 2024-09-16T13:01:15.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934471+0000", 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6040999999999998e-05 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934571+0000", 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100738 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.708 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254153 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.803914883000001, 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083065172000000007, 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254153 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:15.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851679+0000", 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851681+0000", 2024-09-16T13:01:15.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.137e-06 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851687+0000", 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.643e-06 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851690+0000", 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5580000000000001e-06 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934597+0000", 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082906881000000002 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934663+0000", 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5828999999999996e-05 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934743+0000", 2024-09-16T13:01:15.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0714e-05 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254155 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.803899195, 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083254350000000005, 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254155 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851697+0000", 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.187e-06 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851702+0000", 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7520000000000002e-06 2024-09-16T13:01:15.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851706+0000", 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7979999999999999e-06 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934769+0000", 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083062834000000002 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934829+0000", 2024-09-16T13:01:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0430999999999998e-05 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934948+0000", 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118905 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254157 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 207872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.80388327, 2024-09-16T13:01:15.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083456394000000003, 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254157 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851712+0000", 2024-09-16T13:01:15.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0150000000000002e-06 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851718+0000", 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5999999999999997e-06 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851722+0000", 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5810000000000001e-06 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934992+0000", 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083270799000000006 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935066+0000", 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3851999999999997e-05 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935166+0000", 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100145 2024-09-16T13:01:15.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254159 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 210944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.803867923999999, 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083625606000000005, 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254159 2024-09-16T13:01:15.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851726+0000", 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851728+0000", 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851734+0000", 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7429999999999999e-06 2024-09-16T13:01:15.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851738+0000", 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9620000000000004e-06 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935195+0000", 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083457518999999994 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935262+0000", 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6465999999999997e-05 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935351+0000", 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9314000000000006e-05 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254161 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 218112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.803851934000001, 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083866861000000001, 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254161 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851742+0000", 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851744+0000", 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1550000000000001e-06 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851749+0000", 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5829999999999997e-06 2024-09-16T13:01:15.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851753+0000", 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6990000000000001e-06 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935377+0000", 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083624025000000005 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935454+0000", 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7168000000000004e-05 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935608+0000", 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153774 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254163 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 221184~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.731 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.803836157999999, 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084055344000000004, 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254163 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:01:15.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851760+0000", 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2249999999999999e-06 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851765+0000", 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7799999999999997e-06 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851769+0000", 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-06 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935654+0000", 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083884829999999994 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935723+0000", 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9163000000000002e-05 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935812+0000", 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9237e-05 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254164 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.803820115000001, 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084726810999999999, 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254164 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851775+0000", 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.945e-06 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851781+0000", 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851785+0000", 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6320000000000001e-06 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936053+0000", 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084268018 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936130+0000", 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7478999999999995e-05 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936500+0000", 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036965099999999997 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254165 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 228352~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.740 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.803804590999999, 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084960919999999995, 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254165 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851791+0000", 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2819999999999999e-06 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851797+0000", 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7969999999999997e-06 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851801+0000", 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6679999999999999e-06 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936554+0000", 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084753203999999999 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936625+0000", 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1406e-05 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936750+0000", 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012416700000000001 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254166 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 6144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.803788780000001, 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.085115273000000005, 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254166 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.745 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:15.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856445+0000", 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046400929999999996 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856451+0000", 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5119999999999997e-06 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856463+0000", 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2252999999999999e-05 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856470+0000", 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4160000000000001e-06 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936767+0000", 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080297258999999996 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936832+0000", 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4685999999999997e-05 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:15.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:15.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:15.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936920+0000", 2024-09-16T13:01:15.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8053999999999994e-05 2024-09-16T13:01:15.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.749 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:15.749 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:15.749 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:15.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:15.751 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:15.748+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:15.771 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /var/log/ceph/exp.4.18.6 2024-09-16T13:01:15.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T13:01:15.851 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:15.848+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:15.938 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:15.938 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:15.938 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:15.950 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:15.951 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:15.948+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:16.043 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:16.043 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:16.043 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:16.043 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:16.043 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:16.053 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:16.053 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:16.044+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:16.141 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:01:16.141 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":129,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T13:01:15.808943+0000","last_up_change":"2024-09-16T13:01:07.749934+0000","last_in_change":"2024-09-16T12:59:39.968761+0000","flags":"noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5806080,"flags_set":["nodeep-scrub","noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T12:58:44.284177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T12:58:51.733915+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T12:59:00.193651+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T12:59:09.779825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":58,"snap_epoch":129,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":22186},{"type":"v1","addr":"172.21.15.42:6819","nonce":22186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":22186},{"type":"v1","addr":"172.21.15.42:6823","nonce":22186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":22186},{"type":"v1","addr":"172.21.15.42:6830","nonce":22186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":22186},{"type":"v1","addr":"172.21.15.42:6826","nonce":22186}]},"public_addr":"172.21.15.42:6819/22186","cluster_addr":"172.21.15.42:6823/22186","heartbeat_back_addr":"172.21.15.42:6830/22186","heartbeat_front_addr":"172.21.15.42:6826/22186","state":["exists","up"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":22184},{"type":"v1","addr":"172.21.15.42:6809","nonce":22184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":22184},{"type":"v1","addr":"172.21.15.42:6811","nonce":22184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":22184},{"type":"v1","addr":"172.21.15.42:6815","nonce":22184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":22184},{"type":"v1","addr":"172.21.15.42:6813","nonce":22184}]},"public_addr":"172.21.15.42:6809/22184","cluster_addr":"172.21.15.42:6811/22184","heartbeat_back_addr":"172.21.15.42:6815/22184","heartbeat_front_addr":"172.21.15.42:6813/22184","state":["exists","up"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":22185},{"type":"v1","addr":"172.21.15.42:6801","nonce":22185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":22185},{"type":"v1","addr":"172.21.15.42:6803","nonce":22185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":22185},{"type":"v1","addr":"172.21.15.42:6807","nonce":22185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":22185},{"type":"v1","addr":"172.21.15.42:6805","nonce":22185}]},"public_addr":"172.21.15.42:6801/22185","cluster_addr":"172.21.15.42:6803/22185","heartbeat_back_addr":"172.21.15.42:6807/22185","heartbeat_front_addr":"172.21.15.42:6805/22185","state":["exists","up"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":22187},{"type":"v1","addr":"172.21.15.42:6818","nonce":22187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":22187},{"type":"v1","addr":"172.21.15.42:6822","nonce":22187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":22187},{"type":"v1","addr":"172.21.15.42:6831","nonce":22187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":22187},{"type":"v1","addr":"172.21.15.42:6827","nonce":22187}]},"public_addr":"172.21.15.42:6818/22187","cluster_addr":"172.21.15.42:6822/22187","heartbeat_back_addr":"172.21.15.42:6831/22187","heartbeat_front_addr":"172.21.15.42:6827/22187","state":["exists","up"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":83,"up_from":108,"up_thru":121,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":33598},{"type":"v1","addr":"172.21.15.136:6817","nonce":33598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":33598},{"type":"v1","addr":"172.21.15.136:6819","nonce":33598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":33598},{"type":"v1","addr":"172.21.15.136:6823","nonce":33598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":33598},{"type":"v1","addr":"172.21.15.136:6821","nonce":33598}]},"public_addr":"172.21.15.136:6817/33598","cluster_addr":"172.21.15.136:6819/33598","heartbeat_back_addr":"172.21.15.136:6823/33598","heartbeat_front_addr":"172.21.15.136:6821/33598","state":["exists","up"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22340},{"type":"v1","addr":"172.21.15.136:6809","nonce":22340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22340},{"type":"v1","addr":"172.21.15.136:6811","nonce":22340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22340},{"type":"v1","addr":"172.21.15.136:6815","nonce":22340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22340},{"type":"v1","addr":"172.21.15.136:6813","nonce":22340}]},"public_addr":"172.21.15.136:6809/22340","cluster_addr":"172.21.15.136:6811/22340","heartbeat_back_addr":"172.21.15.136:6815/22340","heartbeat_front_addr":"172.21.15.136:6813/22340","state":["exists","up"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":121,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22364},{"type":"v1","addr":"172.21.15.136:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":22364},{"type":"v1","addr":"172.21.15.136:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":22364},{"type":"v1","addr":"172.21.15.136:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22364},{"type":"v1","addr":"172.21.15.136:6829","nonce":22364}]},"public_addr":"172.21.15.136:6825/22364","cluster_addr":"172.21.15.136:6827/22364","heartbeat_back_addr":"172.21.15.136:6831/22364","heartbeat_front_addr":"172.21.15.136:6829/22364","state":["exists"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22341},{"type":"v1","addr":"172.21.15.136:6801","nonce":22341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22341},{"type":"v1","addr":"172.21.15.136:6803","nonce":22341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22341},{"type":"v1","addr":"172.21.15.136:6807","nonce":22341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22341},{"type":"v1","addr":"172.21.15.136:6805","nonce":22341}]},"public_addr":"172.21.15.136:6801/22341","cluster_addr":"172.21.15.136:6803/22341","heartbeat_back_addr":"172.21.15.136:6807/22341","heartbeat_front_addr":"172.21.15.136:6805/22341","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.264463+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.214474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.217927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.435822+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.451286+0000","dead_epoch":84},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.287403+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.440296+0000","dead_epoch":121},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.140630+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"4.12","osds":[5,1]}],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":1,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":25,"length":8}]}],"new_removed_snaps":[{"pool":4,"snaps":[{"begin":25,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T13:01:16.149 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:16.149 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:16.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:16.149 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:16.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.149 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260177 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.149 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:16.149 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.068020283000003, 2024-09-16T13:01:16.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039279541000000001, 2024-09-16T13:01:16.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260177 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075845+0000", 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075847+0000", 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:16.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075852+0000", 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.507e-06 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075856+0000", 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6389999999999998e-06 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.114965+0000", 2024-09-16T13:01:16.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039109496000000001 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115034+0000", 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8378000000000002e-05 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115124+0000", 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0334999999999995e-05 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260178 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.068004887000001, 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039530044, 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260178 2024-09-16T13:01:16.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075866+0000", 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0909999999999998e-06 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075872+0000", 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.139e-06 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075876+0000", 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0419999999999996e-06 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115194+0000", 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039317193 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115259+0000", 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5399000000000002e-05 2024-09-16T13:01:16.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115390+0000", 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130803 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260179 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067984037000002, 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039926426000000001, 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260179 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075883+0000", 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6959999999999999e-06 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075889+0000", 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9479999999999996e-06 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075893+0000", 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7840000000000001e-06 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115419+0000", 2024-09-16T13:01:16.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039526517999999997 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115577+0000", 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015783999999999999 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115807+0000", 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023010399999999999 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.162 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260180 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067968290000003, 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040245398000000002, 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.163 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260180 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075897+0000", 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075899+0000", 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8169999999999999e-06 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075904+0000", 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-16T13:01:16.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075908+0000", 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5889999999999999e-06 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115837+0000", 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039929193000000002 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115978+0000", 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014115800000000001 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116142+0000", 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016352900000000001 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260181 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 180224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.167 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067953082999999, 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040482388000000001, 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260181 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075914+0000", 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6759999999999999e-06 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075919+0000", 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2360000000000004e-06 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075922+0000", 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-16T13:01:16.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116171+0000", 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040248702999999997 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116239+0000", 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8391000000000003e-05 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116394+0000", 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000154575 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260182 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067938372, 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042150168000000002, 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260182 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:16.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075927+0000", 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075929+0000", 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7859999999999999e-06 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075934+0000", 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7679999999999997e-06 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075938+0000", 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2590000000000001e-06 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116426+0000", 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040488077999999997 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116493+0000", 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7681000000000005e-05 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118077+0000", 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015832260000000001 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.176 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260183 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067923383999997, 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040886082999999997, 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260183 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075942+0000", 2024-09-16T13:01:16.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075943+0000", 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.778e-06 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075949+0000", 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075953+0000", 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116672+0000", 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040719563 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116743+0000", 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0729000000000007e-05 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116827+0000", 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4364999999999996e-05 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260184 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067908221000003, 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041043524999999997, 2024-09-16T13:01:16.181 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260184 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075959+0000", 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6869999999999999e-06 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075964+0000", 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.242e-06 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075967+0000", 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116850+0000", 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040882977000000001 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116916+0000", 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5413000000000005e-05 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117000+0000", 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4431999999999994e-05 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.185 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260185 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067892981999996, 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041177895999999999, 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.186 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260185 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075974+0000", 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0049999999999999e-06 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075980+0000", 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7470000000000002e-06 2024-09-16T13:01:16.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075984+0000", 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117022+0000", 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041038789999999999 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117090+0000", 2024-09-16T13:01:16.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7712000000000005e-05 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117150+0000", 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9542e-05 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260257 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067851128000001, 2024-09-16T13:01:16.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041222406000000003, 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260257 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076017+0000", 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4990000000000001e-06 2024-09-16T13:01:16.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076024+0000", 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076028+0000", 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9319999999999998e-06 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117162+0000", 2024-09-16T13:01:16.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04113472 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117199+0000", 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6507999999999997e-05 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117236+0000", 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7023000000000001e-05 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260258 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067832609, 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041343291999999997, 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260258 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:16.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076035+0000", 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.058e-06 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076041+0000", 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.998e-06 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076044+0000", 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3900000000000002e-06 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117256+0000", 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041212555999999997 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117323+0000", 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6568000000000004e-05 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117375+0000", 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2361999999999998e-05 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260259 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067817191000003, 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041437587999999997, 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260259 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5499999999999999e-07 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076050+0000", 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076055+0000", 2024-09-16T13:01:16.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5609999999999998e-06 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076059+0000", 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117396+0000", 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041336879999999999 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117454+0000", 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8443999999999998e-05 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117485+0000", 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1047999999999998e-05 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260260 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067802391999997, 2024-09-16T13:01:16.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041496984000000001, 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260260 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076063+0000", 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076070+0000", 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7919999999999993e-06 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076076+0000", 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9359999999999996e-06 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076080+0000", 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8240000000000001e-06 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117497+0000", 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041416896000000002 2024-09-16T13:01:16.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117531+0000", 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4e-05 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117559+0000", 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8177000000000001e-05 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260261 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067780155999998, 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041555793000000001, 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260261 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076087+0000", 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7850000000000001e-06 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076093+0000", 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6879999999999996e-06 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076096+0000", 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T13:01:16.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117570+0000", 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041474329999999997 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117611+0000", 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0852999999999997e-05 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117640+0000", 2024-09-16T13:01:16.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9213e-05 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260262 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067764709999999, 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041643056999999997, 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260262 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076102+0000", 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8509999999999999e-06 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076108+0000", 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8560000000000003e-06 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076111+0000", 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0620000000000001e-06 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117651+0000", 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041539980999999997 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117703+0000", 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1975000000000003e-05 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117743+0000", 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9872e-05 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260263 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067749728999999, 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041724412000000002, 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260263 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:16.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076117+0000", 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.762e-06 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076123+0000", 2024-09-16T13:01:16.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076126+0000", 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.427e-06 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117757+0000", 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041631230999999998 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117792+0000", 2024-09-16T13:01:16.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4078999999999997e-05 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117839+0000", 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7861000000000001e-05 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260264 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067734883, 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041783638999999997, 2024-09-16T13:01:16.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260264 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076132+0000", 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.751e-06 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076138+0000", 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6729999999999997e-06 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076141+0000", 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117851+0000", 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04171064 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117885+0000", 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3807999999999999e-05 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117913+0000", 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8229999999999999e-05 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260265 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067720280000003, 2024-09-16T13:01:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042095026000000001, 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260265 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076145+0000", 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076147+0000", 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076152+0000", 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5130000000000004e-06 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076156+0000", 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117924+0000", 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041768179000000002 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117980+0000", 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6125000000000002e-05 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118239+0000", 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025934200000000003 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260266 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067705412000002, 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045273031999999998, 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260266 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076160+0000", 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076161+0000", 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.796e-06 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076167+0000", 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-06 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076171+0000", 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.579e-06 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.120966+0000", 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.044795297999999997 2024-09-16T13:01:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121159+0000", 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192618 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121432+0000", 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000273663 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260267 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.067689035000001, 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046181535000000003, 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260267 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076178+0000", 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7969999999999999e-06 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076184+0000", 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6570000000000002e-06 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076187+0000", 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.455e-06 2024-09-16T13:01:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122253+0000", 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046065813999999997 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122318+0000", 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5338000000000004e-05 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122357+0000", 2024-09-16T13:01:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9088999999999998e-05 2024-09-16T13:01:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.238 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:16.240 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T12:58:44.284177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T12:58:51.733915+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T12:59:00.193651+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T12:59:09.779825+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 58, 'snap_epoch': 129, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-09-16T13:01:16.240 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool .mgr 2024-09-16T13:01:16.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:16.242 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:16.236+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:16.341 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:16.336+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:16.343 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:16.343 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:16.352 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:16.441 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:16.436+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:16.447 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:16.447 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:16.447 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:16.447 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:16.447 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:16.457 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:16.542 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:16.536+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:16.562 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.488466918999997, 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:01:16.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.488414499999998, 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:01:16.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.488397652000003, 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:01:16.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.488381537999999, 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.576 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:16.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.488316607999998, 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:16.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:01:16.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.488300508999998, 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:16.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.488226728000001, 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.590 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:16.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.488209527000002, 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:16.598 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.488192824000002, 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:01:16.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.951881569000001, 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:01:16.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:01:16.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.951864432999997, 2024-09-16T13:01:16.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.951844850999997, 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.951813991999998, 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:01:16.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.951745895999998, 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:16.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.944442041000002, 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.626 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:16.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.944424452, 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:16.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:16.634 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.944409559, 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:01:16.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.944377942000003, 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:01:16.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.944359921, 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.647 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.030324548999999, 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2569454119999999, 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 279128 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527093+0000", 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4040000000000002e-06 2024-09-16T13:01:16.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527104+0000", 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-05 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527133+0000", 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8909999999999999e-05 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:16.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527144+0000", 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1497e-05 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527223+0000", 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8577000000000003e-05 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527236+0000", 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3362e-05 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783387+0000", 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2561508619999999 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783699+0000", 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311687 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.784036+0000", 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033771300000000001 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.652 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:16.653 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:16.653 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:16.654 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:16.655 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:16.648+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:16.755 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:16.752+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:16.760 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:16.760 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:16.760 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:16.760 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:16.769 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:16.838 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T13:01:16.856 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:16.852+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:16.871 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:16.871 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:16.880 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:16.956 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:16.952+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:17.039 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:17.039 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:17.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:17.039 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:17.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.039 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240664 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 152576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.039 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.492287134000001, 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047790817999999999, 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240664 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:17.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1900000000000003e-07 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543064+0000", 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4449999999999999e-06 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543079+0000", 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5221e-05 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543084+0000", 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8590000000000004e-06 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590750+0000", 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047665710999999999 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590801+0000", 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1827999999999999e-05 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590852+0000", 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0235000000000003e-05 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240665 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.492257731000002, 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047896192999999997, 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.044 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240665 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543091+0000", 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543094+0000", 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7149999999999998e-06 2024-09-16T13:01:17.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543102+0000", 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6549999999999993e-06 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543108+0000", 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4840000000000003e-06 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590885+0000", 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047777396 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590936+0000", 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0611e-05 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590987+0000", 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0785000000000003e-05 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240666 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 162816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.492235463, 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048010069000000002, 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.049 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240666 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8500000000000002e-07 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543116+0000", 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.531e-06 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543123+0000", 2024-09-16T13:01:17.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1069999999999999e-06 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543127+0000", 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4309999999999996e-06 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591020+0000", 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047892413000000002 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591071+0000", 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1650000000000002e-05 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591123+0000", 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1452000000000001e-05 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240667 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.053 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.492216681000002, 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048125951, 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240667 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543132+0000", 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5099999999999999e-07 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543134+0000", 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1500000000000002e-06 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543141+0000", 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543146+0000", 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9200000000000003e-06 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591158+0000", 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048011246 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591207+0000", 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9295999999999997e-05 2024-09-16T13:01:17.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591257+0000", 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0426000000000001e-05 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240668 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 173056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.492196976000002, 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048243280999999999, 2024-09-16T13:01:17.058 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240668 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543152+0000", 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543154+0000", 2024-09-16T13:01:17.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5380000000000001e-06 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543161+0000", 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7920000000000004e-06 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543172+0000", 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1327e-05 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591290+0000", 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048118181000000003 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591344+0000", 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4061000000000003e-05 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591394+0000", 2024-09-16T13:01:17.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9866e-05 2024-09-16T13:01:17.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.063 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240766 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.490992650999999, 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047716644000000003, 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240766 2024-09-16T13:01:17.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544356+0000", 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2500000000000005e-07 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544373+0000", 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6685999999999999e-05 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544381+0000", 2024-09-16T13:01:17.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7190000000000001e-06 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544384+0000", 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591969+0000", 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047585483999999997 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592021+0000", 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1839000000000003e-05 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592072+0000", 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754000000000002e-05 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240767 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 201728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.490960776000001, 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047822479000000001, 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240767 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.069 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544388+0000", 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.51e-07 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544390+0000", 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.824e-06 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544393+0000", 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.681e-06 2024-09-16T13:01:17.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544396+0000", 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4940000000000002e-06 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592105+0000", 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047709184000000002 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592158+0000", 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2772999999999998e-05 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592210+0000", 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2071999999999999e-05 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240768 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.073 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.490947974999997, 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047944647, 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240768 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:17.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0400000000000002e-07 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544403+0000", 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1430000000000001e-06 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544406+0000", 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7450000000000001e-06 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544409+0000", 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592241+0000", 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047832146999999998 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592291+0000", 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9289000000000002e-05 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592345+0000", 2024-09-16T13:01:17.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4067000000000003e-05 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240769 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.490936349999998, 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048066225999999997, 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.078 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240769 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1399999999999998e-07 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544413+0000", 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.42e-06 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544417+0000", 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.529e-06 2024-09-16T13:01:17.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544420+0000", 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5620000000000002e-06 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592378+0000", 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047958354000000002 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592427+0000", 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9013e-05 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592478+0000", 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1034e-05 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240770 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.490926086999998, 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048189432999999997, 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240770 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.083 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3200000000000001e-07 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544424+0000", 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4449999999999999e-06 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544427+0000", 2024-09-16T13:01:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5049999999999999e-06 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544430+0000", 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.61e-06 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592511+0000", 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048080912000000003 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592563+0000", 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2107000000000001e-05 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592611+0000", 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8522000000000001e-05 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240771 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.490891480000002, 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290128000000002, 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240771 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5700000000000002e-07 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544459+0000", 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6840000000000001e-06 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544466+0000", 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999997e-06 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544469+0000", 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592645+0000", 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048175216999999999 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592696+0000", 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1292999999999998e-05 2024-09-16T13:01:17.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592747+0000", 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0751000000000002e-05 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240772 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.490868184999997, 2024-09-16T13:01:17.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048403333999999999, 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240772 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544482+0000", 2024-09-16T13:01:17.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8390000000000001e-06 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544486+0000", 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9369999999999997e-06 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544489+0000", 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8459999999999999e-06 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592779+0000", 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290301000000001 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592831+0000", 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2335e-05 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592883+0000", 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1786999999999997e-05 2024-09-16T13:01:17.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240773 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.490856800000003, 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048527963, 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240773 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:17.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544492+0000", 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.84e-07 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544493+0000", 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6279999999999999e-06 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544498+0000", 2024-09-16T13:01:17.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4689999999999999e-06 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544500+0000", 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592916+0000", 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048415672 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592965+0000", 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9120999999999999e-05 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593019+0000", 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3950999999999999e-05 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.101 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240774 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.490845145000002, 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048649546000000002, 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240774 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544505+0000", 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.61e-06 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544508+0000", 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4929999999999998e-06 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544511+0000", 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4169999999999999e-06 2024-09-16T13:01:17.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593051+0000", 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048539604 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593103+0000", 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2068999999999999e-05 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593152+0000", 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9870999999999997e-05 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240775 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.490834890999999, 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048773328999999997, 2024-09-16T13:01:17.106 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240775 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545618+0000", 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011045810000000001 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545625+0000", 2024-09-16T13:01:17.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6079999999999998e-06 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545644+0000", 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.823e-05 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545653+0000", 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3000000000000007e-06 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593186+0000", 2024-09-16T13:01:17.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047533387000000003 2024-09-16T13:01:17.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593235+0000", 2024-09-16T13:01:17.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8767000000000003e-05 2024-09-16T13:01:17.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593286+0000", 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1456000000000003e-05 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240776 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.489682752999997, 2024-09-16T13:01:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047754889000000002, 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240776 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545666+0000", 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-06 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545671+0000", 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2270000000000004e-06 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545681+0000", 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0506000000000001e-05 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545687+0000", 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0970000000000002e-06 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593320+0000", 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047632999000000002 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593369+0000", 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8882999999999999e-05 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593420+0000", 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1127e-05 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240777 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.489654131999998, 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047878318000000003, 2024-09-16T13:01:17.116 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240777 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545695+0000", 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545698+0000", 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0709999999999999e-06 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545707+0000", 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7629999999999995e-06 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545713+0000", 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2839999999999997e-06 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593453+0000", 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047740281000000002 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593518+0000", 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5493000000000005e-05 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593572+0000", 2024-09-16T13:01:17.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3739000000000002e-05 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240778 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.489629692999998, 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047999411999999998, 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240778 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545719+0000", 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.13e-07 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545728+0000", 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1560000000000001e-06 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545901+0000", 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017311599999999999 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545909+0000", 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.028e-06 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593606+0000", 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047696660000000002 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593666+0000", 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9611999999999998e-05 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593718+0000", 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2227e-05 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240951 4.8 4:10cc0d5f:::rbd_data.11b552946c0f.0000000000000120:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.481549424000001, 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048226894999999999, 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.125 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240951 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:17.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2300000000000002e-07 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553801+0000", 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.686e-06 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553805+0000", 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3039999999999998e-06 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553809+0000", 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.599994+0000", 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046185154999999999 2024-09-16T13:01:17.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.600112+0000", 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118589 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.602026+0000", 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019131370000000001 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.129 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:272626 4.12 4:4935e950:::rbd_data.11b59088eb24.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.577521802, 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022439990000001, 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 272626 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457828+0000", 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457835+0000", 2024-09-16T13:01:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7900000000000002e-06 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457853+0000", 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7671000000000001e-05 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457859+0000", 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2949999999999999e-06 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457913+0000", 2024-09-16T13:01:17.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4112e-05 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457922+0000", 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0850000000000003e-06 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.457879+0000", 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.99995641700000004 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:17.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458079+0000", 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020072300000000001 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458304+0000", 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000224601 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459393+0000", 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010886170000000001 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459877+0000", 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048443500000000002 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460006+0000", 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012914 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460070+0000", 2024-09-16T13:01:17.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4074999999999999e-05 2024-09-16T13:01:17.136 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.136 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.136 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.136 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:17.136 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:17.136 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:17.136 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:01:17.138 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:17.132+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:17.138 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:17.238 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:17.232+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:17.246 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:17.246 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:17.246 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:17.246 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:17.255 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:17.281 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:17.302 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:17.324 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:17.340 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:17.332+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:17.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:17.359 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:17.359 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:17.359 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:17.359 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:17.359 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:17.367 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:17.374 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:17.377 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2024-09-16T13:01:17.388 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:17.411 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:17.416 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:17.416 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:17.440 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:17.432+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:17.467 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:17.467 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:17.476 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:17.542 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:17.536+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:17.575 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:17.575 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:17.584 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:17.643 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:17.636+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:17.689 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:17.689 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:17.698 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:17.744 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:17.740+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:17.800 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:17.801 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:17.810 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:17.845 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:17.840+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:17.853 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T13:01:17.934 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:17.935 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:17.941 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool rbd 2024-09-16T13:01:17.946 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:17.946 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:17.940+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:18.046 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:18.044+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:18.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:18.148 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:18.144+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:18.159 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:18.159 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:18.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:18.248 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:18.240+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:18.266 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:18.266 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:18.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:18.349 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:18.344+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:18.378 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:18.378 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:18.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:18.450 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:18.444+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:18.490 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:18.490 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:18.499 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:18.551 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:18.544+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:18.604 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:18.604 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:18.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:18.657 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:18.652+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:18.753 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:18.748+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:18.760 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:18.760 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:18.769 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:18.854 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:18.848+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:18.884 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:18.884 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:18.893 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:18.954 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:18.948+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:18.992 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:18.992 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:19.002 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:19.055 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:19.048+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:19.087 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2024-09-16T13:01:19.105 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:19.105 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:19.115 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:19.156 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:19.148+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:19.211 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:19.211 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:19.221 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:19.267 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:19.264+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:19.329 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:19.329 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:19.340 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:19.358 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:19.352+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:19.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:01:19.439 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:19.439 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:19.448 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:19.459 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:19.452+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:19.553 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:19.553 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:19.560 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:19.552+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:19.562 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:19.573 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:19.597 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:19.623 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:19.645 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool cache 2024-09-16T13:01:19.648 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:19.660 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:19.652+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:19.663 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:19.664 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:19.668 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:19.673 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:19.691 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:19.718 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:19.722 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:19.723 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:19.765 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:19.760+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:19.815 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:19.815 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:19.815 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:19.815 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:19.815 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:19.824 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:19.862 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:19.856+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:19.872 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 154.37738864400001, 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:01:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.55227159699999, 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:01:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.552224341, 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:01:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.552021405, 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.55199014300001, 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:01:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:01:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:01:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:01:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:01:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:01:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:01:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:01:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:01:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:01:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.551954381, 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:01:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:01:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.55175975099999, 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:01:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.55171290600001, 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:01:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:01:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:01:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.007436735, 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:01:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:01:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.007382828, 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:01:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:01:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:01:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:01:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:01:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:01:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.00734052300001, 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:01:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:01:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:01:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.00718209799999, 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:01:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:01:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:01:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.007132617, 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:01:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.00700204699999, 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:01:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.00682053400001, 2024-09-16T13:01:20.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:20.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:20.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:20.023 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 152.19629286700001, 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:01:20.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:01:20.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:20.029 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 152.19614391299999, 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:20.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:01:20.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64507 4.1e e83/31 4:7f7c7834:::rbd_data.11b5c2ee3542.0000000000000008:head v 83'225, mlcod=81'223)", 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:20.035 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 61.156730625999998, 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021284401000000001, 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770380+0000", 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8999999999999998e-07 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777141+0000", 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067617670000000001 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777157+0000", 2024-09-16T13:01:20.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777167+0000", 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777267+0000", 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100364 2024-09-16T13:01:20.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777304+0000", 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7165000000000001e-05 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791601+0000", 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014297275999999999 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791664+0000", 2024-09-16T13:01:20.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2217000000000004e-05 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64509 4.15 e83/31 4:ad2e18f4:::rbd_data.11b5c2ee3542.0000000000000009:head v 83'183, mlcod=81'180)", 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 61.149972724000001, 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014054395000000001, 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777138+0000", 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T13:01:20.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778534+0000", 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013956769999999999 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778555+0000", 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1046000000000001e-05 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778564+0000", 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9290000000000004e-06 2024-09-16T13:01:20.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778633+0000", 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9363000000000007e-05 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778666+0000", 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135000000000003e-05 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791145+0000", 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012479743 2024-09-16T13:01:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791192+0000", 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6156000000000003e-05 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:217754 4.17 e92/31 4:ee4048fa:::rbd_data.11b58362ce83.0000000000000002:head v 92'697, mlcod=91'693)", 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.427802669999998, 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011416822, 2024-09-16T13:01:20.044 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499309+0000", 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6929999999999999e-06 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499322+0000", 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499354+0000", 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135999999999999e-05 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499365+0000", 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111e-05 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499504+0000", 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013877599999999999 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499565+0000", 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1939000000000004e-05 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510591+0000", 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011025874999999999 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510724+0000", 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013266299999999999 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.048 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:20.050 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:20.050 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:20.044+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:20.151 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:20.148+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:20.155 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:20.156 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:20.156 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:20.156 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:20.164 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:20.252 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:20.243+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:20.262 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:20.262 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:20.262 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:20.262 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:20.262 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:20.271 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:20.353 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:20.348+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:20.371 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-16T13:01:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.831520482999998, 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98766548899999995, 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279128 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:20.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534648+0000", 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1459999999999999e-06 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534655+0000", 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5220000000000004e-06 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534674+0000", 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9717000000000001e-05 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534683+0000", 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534759+0000", 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6074000000000004e-05 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534770+0000", 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0665999999999999e-05 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522281+0000", 2024-09-16T13:01:20.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98751079600000002 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522312+0000", 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0876000000000001e-05 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.9 109/108 [PushOp(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29, version: 106'278, data_included: [0~688128,802816~10077,884736~295,1950271~32193], data_size: 730693, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29@106'278, size: 1982464, copy_subset: [0~1982464], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1982464, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:20.376 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.724501351000001, 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067875690000000002, 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641666+0000", 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3060000000000001e-06 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644856+0000", 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00318995 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644894+0000", 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7382999999999998e-05 2024-09-16T13:01:20.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644913+0000", 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9415999999999998e-05 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645084+0000", 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017036899999999999 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645106+0000", 2024-09-16T13:01:20.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2887e-05 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.648453+0000", 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003346258 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279882 4.c 4:3656acfa:::rbd_data.11b53ee6e6bb.0000000000000009:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.497524187, 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004921407, 2024-09-16T13:01:20.380 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279882 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868642+0000", 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868645+0000", 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5550000000000001e-06 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868656+0000", 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1358999999999999e-05 2024-09-16T13:01:20.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868691+0000", 2024-09-16T13:01:20.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4459999999999999e-05 2024-09-16T13:01:20.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868703+0000", 2024-09-16T13:01:20.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2556e-05 2024-09-16T13:01:20.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.868818+0000", 2024-09-16T13:01:20.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011502000000000001 2024-09-16T13:01:20.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.869209+0000", 2024-09-16T13:01:20.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039085699999999999 2024-09-16T13:01:20.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:20.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.869563+0000", 2024-09-16T13:01:20.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035391699999999998 2024-09-16T13:01:20.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.871027+0000", 2024-09-16T13:01:20.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00146393 2024-09-16T13:01:20.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873450+0000", 2024-09-16T13:01:20.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002422914 2024-09-16T13:01:20.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873527+0000", 2024-09-16T13:01:20.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7286000000000005e-05 2024-09-16T13:01:20.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:02.873564+0000", 2024-09-16T13:01:20.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6553000000000001e-05 2024-09-16T13:01:20.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.388 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.389 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280527 4.13 4:cc227b3c:::rbd_data.11b5cb8797c6.0000000000000000:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:20.389 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:20.389 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.493041994, 2024-09-16T13:01:20.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052047300000000003, 2024-09-16T13:01:20.389 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.389 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.390 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.390 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.390 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.390 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280527 2024-09-16T13:01:20.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.390 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:20.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:20.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873126+0000", 2024-09-16T13:01:20.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6530000000000001e-06 2024-09-16T13:01:20.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873133+0000", 2024-09-16T13:01:20.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0940000000000002e-06 2024-09-16T13:01:20.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873153+0000", 2024-09-16T13:01:20.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9538000000000002e-05 2024-09-16T13:01:20.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873161+0000", 2024-09-16T13:01:20.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1009999999999992e-06 2024-09-16T13:01:20.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873245+0000", 2024-09-16T13:01:20.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4075999999999999e-05 2024-09-16T13:01:20.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873360+0000", 2024-09-16T13:01:20.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114951 2024-09-16T13:01:20.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:20.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873530+0000", 2024-09-16T13:01:20.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016994699999999999 2024-09-16T13:01:20.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.877617+0000", 2024-09-16T13:01:20.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040873000000000003 2024-09-16T13:01:20.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878160+0000", 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054299199999999995 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878251+0000", 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1062000000000003e-05 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878329+0000", 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8016000000000007e-05 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280530 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.49297999, 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056839330000000004, 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280530 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8199999999999999e-07 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873191+0000", 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4599999999999999e-06 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873198+0000", 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8399999999999997e-06 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873201+0000", 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.641e-06 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873298+0000", 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6682999999999996e-05 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873451+0000", 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015304000000000001 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873692+0000", 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024124300000000001 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.876380+0000", 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026873980000000001 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878771+0000", 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002391013 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878824+0000", 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3136000000000001e-05 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878870+0000", 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6796999999999999e-05 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280650 4.1b 4:d8560506:::rbd_header.11b5de89dd5b:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.60665558, 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085055370000000005, 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280650 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759514+0000", 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7429999999999998e-06 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759524+0000", 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8309999999999996e-06 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759540+0000", 2024-09-16T13:01:20.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6506000000000001e-05 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759548+0000", 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1810000000000001e-06 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.761995+0000", 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002446518 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.762120+0000", 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125124 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.762945+0000", 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082531599999999998 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.766013+0000", 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030673549999999999 2024-09-16T13:01:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.767864+0000", 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001851161 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.767939+0000", 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5012000000000006e-05 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.768016+0000", 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7789999999999999e-05 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280740 4.18 4:1df486ae:::rbd_data.11b5de89dd5b.0000000000000000:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.457163451, 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070806940000000002, 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.410 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280740 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:20.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909005+0000", 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.317e-06 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909016+0000", 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0852999999999999e-05 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909052+0000", 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5256000000000003e-05 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909064+0000", 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2413e-05 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909168+0000", 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103615 2024-09-16T13:01:20.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909335+0000", 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016715100000000001 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909599+0000", 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000264391 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.914767+0000", 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051679309999999997 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915839+0000", 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001072415 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915986+0000", 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014609099999999999 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.916084+0000", 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8261e-05 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280741 4.13 4:ce26f199:::rbd_data.11b5de89dd5b.0000000000000001:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.457076862999999, 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062230289999999997, 2024-09-16T13:01:20.416 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280741 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909091+0000", 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0100000000000001e-06 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909096+0000", 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4399999999999996e-06 2024-09-16T13:01:20.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909107+0000", 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.066e-05 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909113+0000", 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9460000000000003e-06 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909206+0000", 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3450999999999997e-05 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909341+0000", 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135267 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909519+0000", 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177603 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.913993+0000", 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044734989999999997 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915108+0000", 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001115001 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915207+0000", 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9562999999999995e-05 2024-09-16T13:01:20.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915313+0000", 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010558899999999999 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281211 4.5 4:a14f106d:::rbd_header.11b515823229:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:20.422 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5685298500000009, 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093462500000000004, 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281211 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:20.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797641+0000", 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4259999999999996e-06 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797662+0000", 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.069e-05 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797705+0000", 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3495999999999998e-05 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797719+0000", 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4003e-05 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797828+0000", 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010857 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797840+0000", 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.235e-05 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.802951+0000", 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051111079999999996 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.803147+0000", 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000196109 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806806+0000", 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003658246 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806865+0000", 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9246999999999997e-05 2024-09-16T13:01:20.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806983+0000", 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011800500000000001 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281307 4.c 4:32a25aa2:::rbd_data.11b515823229.000000000000002b:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.924498+0000", 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4416682170000001, 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048053389999999996, 2024-09-16T13:01:20.428 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281307 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924498+0000", 2024-09-16T13:01:20.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924498+0000", 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924499+0000", 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8299999999999997e-07 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924502+0000", 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924510+0000", 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6320000000000008e-06 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924515+0000", 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4710000000000001e-06 2024-09-16T13:01:20.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924648+0000", 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013367300000000001 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924794+0000", 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014601600000000001 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924905+0000", 2024-09-16T13:01:20.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111082 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.927599+0000", 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026937440000000001 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929141+0000", 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015415069999999999 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929229+0000", 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7967000000000007e-05 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929304+0000", 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5094999999999998e-05 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.434 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281308 4.1b 4:db2b1562:::rbd_data.11b515823229.000000000000002f:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4416308390000001, 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056950619999999999, 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281308 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7999999999999995e-07 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924539+0000", 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9799999999999998e-06 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924546+0000", 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4620000000000001e-06 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924550+0000", 2024-09-16T13:01:20.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0040000000000001e-06 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924628+0000", 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7749000000000004e-05 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924853+0000", 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000225491 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:20.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.925042+0000", 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018871099999999999 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928196+0000", 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031538120000000002 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930092+0000", 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001896556 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930167+0000", 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5111999999999995e-05 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930231+0000", 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3504999999999996e-05 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281318 4.e 4:7158d438:::rbd_data.11b515823229.0000000000000081:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:20.440 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4367449390000004, 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058318850000000002, 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281318 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929423+0000", 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2139999999999999e-06 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929428+0000", 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.826e-06 2024-09-16T13:01:20.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929439+0000", 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1549000000000001e-05 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929444+0000", 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0050000000000004e-06 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929506+0000", 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1927999999999994e-05 2024-09-16T13:01:20.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929632+0000", 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012569299999999999 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935090+0000", 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054580920000000003 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935154+0000", 2024-09-16T13:01:20.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3688000000000004e-05 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935253+0000", 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9889999999999994e-05 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281319 4.1b 4:d932f5f4:::rbd_data.11b515823229.0000000000000082:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.930789+0000", 2024-09-16T13:01:20.445 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.4353770749999999, 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044561430000000001, 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281319 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930789+0000", 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930789+0000", 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930791+0000", 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2270000000000001e-06 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930797+0000", 2024-09-16T13:01:20.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9519999999999999e-06 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930813+0000", 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6164999999999999e-05 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930820+0000", 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.289e-06 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930873+0000", 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2984e-05 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.931016+0000", 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142507 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.931134+0000", 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011879699999999999 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.933761+0000", 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026264249999999999 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935133+0000", 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013722300000000001 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935195+0000", 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1925000000000001e-05 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935246+0000", 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0642000000000001e-05 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281500 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:head [delete] snapc 1d=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:11.928207+0000", 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.4379598760000007, 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047239270000000002, 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.451 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281500 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928207+0000", 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928207+0000", 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928208+0000", 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6139999999999999e-06 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928217+0000", 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5569999999999996e-06 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928239+0000", 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2357e-05 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928249+0000", 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8079999999999999e-06 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928339+0000", 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0224999999999998e-05 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928499+0000", 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015945600000000001 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.928838+0000", 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000339792 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.930649+0000", 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001810063 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.932806+0000", 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021571379999999999 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.932879+0000", 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3543999999999999e-05 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:11.932931+0000", 2024-09-16T13:01:20.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1372999999999998e-05 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282172 4.13 4:c99eeb3d:::rbd_data.11b5b3dc7343.0000000000000001:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.942314+0000", 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.42385275, 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005440907, 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.457 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282172 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942314+0000", 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942314+0000", 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942316+0000", 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.176e-06 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942326+0000", 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0514e-05 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942361+0000", 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4875000000000001e-05 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942374+0000", 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2924e-05 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942515+0000", 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140746 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.943277+0000", 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076154200000000003 2024-09-16T13:01:20.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947497+0000", 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042207939999999999 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947593+0000", 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5166000000000003e-05 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947755+0000", 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016217 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282176 4.1 4:82eb3962:::rbd_data.11b5b3dc7343.0000000000000005:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.942450+0000", 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4237166529999996, 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051970929999999999, 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282176 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942450+0000", 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942450+0000", 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942451+0000", 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1230000000000001e-06 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942457+0000", 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8370000000000002e-06 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942474+0000", 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6663e-05 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942484+0000", 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0462000000000001e-05 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942618+0000", 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134167 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.943112+0000", 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049358500000000005 2024-09-16T13:01:20.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.943411+0000", 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000299296 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947145+0000", 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037339589999999998 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947437+0000", 2024-09-16T13:01:20.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029157299999999999 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947547+0000", 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110287 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947647+0000", 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100141 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.467 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282181 4.d 4:b2f030fd:::rbd_data.11b5b3dc7343.0000000000000034:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.946061+0000", 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4201058690000004, 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066091789999999997, 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282181 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946061+0000", 2024-09-16T13:01:20.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946061+0000", 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946064+0000", 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0359999999999998e-06 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946076+0000", 2024-09-16T13:01:20.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2614e-05 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946100+0000", 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3198999999999999e-05 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946115+0000", 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5953000000000001e-05 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946312+0000", 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019610499999999999 2024-09-16T13:01:20.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946957+0000", 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00064557500000000001 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947239+0000", 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028224599999999999 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.949960+0000", 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027203399999999999 2024-09-16T13:01:20.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952517+0000", 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025567709999999998 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952601+0000", 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4557999999999999e-05 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952670+0000", 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8782000000000007e-05 2024-09-16T13:01:20.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282185 4.5 4:a4a7c469:::rbd_data.11b5b3dc7343.0000000000000076:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.948283+0000", 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4178835650000003, 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046425670000000002, 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282185 2024-09-16T13:01:20.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948283+0000", 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948283+0000", 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948286+0000", 2024-09-16T13:01:20.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.683e-06 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948298+0000", 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1946e-05 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948324+0000", 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5999999999999998e-05 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948337+0000", 2024-09-16T13:01:20.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3152999999999999e-05 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948514+0000", 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017744200000000001 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948988+0000", 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047350300000000001 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952690+0000", 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037026030000000001 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952745+0000", 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5179000000000001e-05 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952926+0000", 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000180058 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.477 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282187 4.5 4:a15e8157:::rbd_data.11b5b3dc7343.0000000000000087:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.948368+0000", 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4177983330000004, 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048875869999999997, 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282187 2024-09-16T13:01:20.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948368+0000", 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948368+0000", 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948369+0000", 2024-09-16T13:01:20.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2860000000000001e-06 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948376+0000", 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0959999999999997e-06 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948393+0000", 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7790000000000001e-05 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.948404+0000", 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1039999999999999e-05 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.949950+0000", 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001545966 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.950299+0000", 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034879699999999997 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.953088+0000", 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027888729999999999 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.953195+0000", 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106916 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.953256+0000", 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0822999999999997e-05 2024-09-16T13:01:20.482 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.483 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.483 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.483 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282774 4.c 4:36b2d631:::rbd_object_map.11b51a3eb68b:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-16T13:01:20.483 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:19.869669+0000", 2024-09-16T13:01:20.483 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.49649775800000001, 2024-09-16T13:01:20.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055610959999999997, 2024-09-16T13:01:20.483 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:20.484 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:20.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:20.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:20.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.484 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282774 2024-09-16T13:01:20.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.484 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:20.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:20.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869669+0000", 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869669+0000", 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869671+0000", 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6240000000000002e-06 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:20.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869684+0000", 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2152e-05 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869715+0000", 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1659000000000003e-05 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869729+0000", 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3563e-05 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869823+0000", 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4333999999999995e-05 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869831+0000", 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5859999999999999e-06 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.872858+0000", 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030270700000000002 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.873002+0000", 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014394599999999999 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.873214+0000", 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021210200000000001 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875001+0000", 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001786852 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875103+0000", 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102058 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875170+0000", 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7019000000000003e-05 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875230+0000", 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0130999999999997e-05 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.489 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:20.490 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:20.490 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:20.491 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:20.488+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:20.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:20.592 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:20.588+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:20.603 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:20.603 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:20.603 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:20.612 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:20.693 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:20.688+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:20.713 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:20.713 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:20.713 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:20.713 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:20.713 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:20.722 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:20.793 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:20.788+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:20.811 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2024-09-16T13:01:20.824 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125690527000003, 2024-09-16T13:01:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:01:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:01:20.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125675501000003, 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:01:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125659479000007, 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125643382999996, 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:01:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:01:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125627492999996, 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:01:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:01:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125611823, 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125596075999994, 2024-09-16T13:01:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125580287999995, 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:01:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:01:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125564965999999, 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:01:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125545040999995, 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:01:20.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:01:20.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125529092999997, 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:01:20.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125513932000004, 2024-09-16T13:01:20.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125498284000003, 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:01:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125483187, 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:01:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:01:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:01:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:01:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125467924999995, 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:01:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:01:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125452179000007, 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125436582000006, 2024-09-16T13:01:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:01:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:01:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:01:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125421278000005, 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:01:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:01:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125405839999999, 2024-09-16T13:01:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:01:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:01:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:01:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.125390127000003, 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:01:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:01:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:01:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:01:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:01:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:01:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:20.915 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:20.917 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:20.918 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T13:01:20.918 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:20.912+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:21.018 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:21.011+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:21.030 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:21.030 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:21.030 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:21.030 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:21.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:21.119 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:21.116+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:21.209 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:21.209 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:21.209 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:21.209 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:21.209 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:21.209 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:21.219 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:21.216+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:21.317 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254136 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 136192~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.463171748000001, 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082717923999999998, 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254136 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849492+0000", 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849494+0000", 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0540000000000002e-06 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849500+0000", 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2899999999999999e-06 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849504+0000", 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8120000000000001e-06 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.931987+0000", 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082483231000000004 2024-09-16T13:01:21.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932061+0000", 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4181e-05 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932209+0000", 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014796399999999999 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254137 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 139264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.463155473, 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082930443000000006, 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254137 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:21.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849510+0000", 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0530000000000001e-06 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849516+0000", 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8359999999999997e-06 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849520+0000", 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932239+0000", 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082719191999999997 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932307+0000", 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7824000000000007e-05 2024-09-16T13:01:21.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932438+0000", 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000131354 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254138 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 146432~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.463139542999997, 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083100513000000001, 2024-09-16T13:01:21.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.328 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.328 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.328 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.328 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254138 2024-09-16T13:01:21.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.328 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7300000000000001e-07 2024-09-16T13:01:21.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849526+0000", 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9089999999999998e-06 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849532+0000", 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6420000000000004e-06 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849535+0000", 2024-09-16T13:01:21.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7740000000000002e-06 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932467+0000", 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082931469999999993 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932535+0000", 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7815e-05 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932624+0000", 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9430000000000003e-05 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254139 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 149504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.463123985999999, 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083267068, 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254139 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849540+0000", 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849542+0000", 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9379999999999999e-06 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849547+0000", 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849551+0000", 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5880000000000002e-06 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932652+0000", 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083101176999999998 2024-09-16T13:01:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932719+0000", 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.703e-05 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932806+0000", 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7340000000000001e-05 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254140 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 156672~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.463108712999997, 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083436222000000004, 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254140 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849557+0000", 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8419999999999999e-06 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849564+0000", 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7079999999999997e-06 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849569+0000", 2024-09-16T13:01:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.386e-06 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932833+0000", 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083264099999999994 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932900+0000", 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7007000000000004e-05 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932991+0000", 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0815000000000004e-05 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254141 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.463090160999997, 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083599532000000004, 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254141 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849574+0000", 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1699999999999998e-07 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849576+0000", 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849582+0000", 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9769999999999997e-06 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849585+0000", 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933018+0000", 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083432372000000005 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933086+0000", 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7862999999999997e-05 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933173+0000", 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7050000000000002e-05 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254142 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.463074011000003, 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083767603999999996, 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254142 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849590+0000", 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.51e-07 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849592+0000", 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0360000000000001e-06 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849598+0000", 2024-09-16T13:01:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4139999999999999e-06 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849602+0000", 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.659e-06 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933199+0000", 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083597150999999995 2024-09-16T13:01:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933267+0000", 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7933000000000002e-05 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933357+0000", 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9859999999999997e-05 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254143 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.463057063000001, 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083932426000000004, 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254143 2024-09-16T13:01:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849607+0000", 2024-09-16T13:01:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:01:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849609+0000", 2024-09-16T13:01:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.165e-06 2024-09-16T13:01:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849615+0000", 2024-09-16T13:01:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2029999999999997e-06 2024-09-16T13:01:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849619+0000", 2024-09-16T13:01:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-16T13:01:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933382+0000", 2024-09-16T13:01:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083763447000000005 2024-09-16T13:01:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933447+0000", 2024-09-16T13:01:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4993000000000006e-05 2024-09-16T13:01:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933539+0000", 2024-09-16T13:01:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1434000000000006e-05 2024-09-16T13:01:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254145 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.463040501000002, 2024-09-16T13:01:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084130242999999993, 2024-09-16T13:01:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254145 2024-09-16T13:01:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851596+0000", 2024-09-16T13:01:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019732550000000001 2024-09-16T13:01:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851602+0000", 2024-09-16T13:01:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5720000000000002e-06 2024-09-16T13:01:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851610+0000", 2024-09-16T13:01:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5529999999999993e-06 2024-09-16T13:01:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851616+0000", 2024-09-16T13:01:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6409999999999999e-06 2024-09-16T13:01:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933580+0000", 2024-09-16T13:01:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.081963854000000003 2024-09-16T13:01:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933653+0000", 2024-09-16T13:01:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3901999999999998e-05 2024-09-16T13:01:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933753+0000", 2024-09-16T13:01:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9465999999999999e-05 2024-09-16T13:01:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254149 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.461023978, 2024-09-16T13:01:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082661199000000005, 2024-09-16T13:01:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254149 2024-09-16T13:01:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851640+0000", 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851642+0000", 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851648+0000", 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9200000000000001e-06 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851652+0000", 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0439999999999997e-06 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934105+0000", 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082453495000000002 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934193+0000", 2024-09-16T13:01:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7433000000000001e-05 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934300+0000", 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107513 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254151 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.461002356999998, 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082910496, 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254151 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1799999999999995e-07 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851664+0000", 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4399999999999999e-06 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851670+0000", 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3509999999999998e-06 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851674+0000", 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.912e-06 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934385+0000", 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082710396000000005 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934471+0000", 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6040999999999998e-05 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934571+0000", 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100738 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254153 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.460984904, 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083065172000000007, 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254153 2024-09-16T13:01:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851679+0000", 2024-09-16T13:01:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851681+0000", 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.137e-06 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851687+0000", 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.643e-06 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851690+0000", 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5580000000000001e-06 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934597+0000", 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082906881000000002 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934663+0000", 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5828999999999996e-05 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934743+0000", 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0714e-05 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254155 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.460969216000002, 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083254350000000005, 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254155 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851697+0000", 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.187e-06 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851702+0000", 2024-09-16T13:01:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7520000000000002e-06 2024-09-16T13:01:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851706+0000", 2024-09-16T13:01:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7979999999999999e-06 2024-09-16T13:01:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934769+0000", 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083062834000000002 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934829+0000", 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0430999999999998e-05 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934948+0000", 2024-09-16T13:01:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118905 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254157 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 207872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.460953291000003, 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083456394000000003, 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254157 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851712+0000", 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0150000000000002e-06 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851718+0000", 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5999999999999997e-06 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851722+0000", 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5810000000000001e-06 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934992+0000", 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083270799000000006 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935066+0000", 2024-09-16T13:01:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3851999999999997e-05 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935166+0000", 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100145 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254159 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 210944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.460937944999998, 2024-09-16T13:01:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083625606000000005, 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254159 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851726+0000", 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851728+0000", 2024-09-16T13:01:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851734+0000", 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7429999999999999e-06 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851738+0000", 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9620000000000004e-06 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935195+0000", 2024-09-16T13:01:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083457518999999994 2024-09-16T13:01:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935262+0000", 2024-09-16T13:01:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6465999999999997e-05 2024-09-16T13:01:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935351+0000", 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9314000000000006e-05 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254161 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 218112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.460921955000003, 2024-09-16T13:01:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083866861000000001, 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254161 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851742+0000", 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851744+0000", 2024-09-16T13:01:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1550000000000001e-06 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851749+0000", 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5829999999999997e-06 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851753+0000", 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6990000000000001e-06 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935377+0000", 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083624025000000005 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935454+0000", 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7168000000000004e-05 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935608+0000", 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153774 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254163 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 221184~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.460906178999998, 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084055344000000004, 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254163 2024-09-16T13:01:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851760+0000", 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2249999999999999e-06 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851765+0000", 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7799999999999997e-06 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851769+0000", 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-06 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935654+0000", 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083884829999999994 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935723+0000", 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9163000000000002e-05 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935812+0000", 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9237e-05 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254164 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.460890136000003, 2024-09-16T13:01:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084726810999999999, 2024-09-16T13:01:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254164 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851775+0000", 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.945e-06 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851781+0000", 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851785+0000", 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6320000000000001e-06 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936053+0000", 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084268018 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936130+0000", 2024-09-16T13:01:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7478999999999995e-05 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936500+0000", 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036965099999999997 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254165 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 228352~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.460874611999998, 2024-09-16T13:01:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084960919999999995, 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254165 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851791+0000", 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2819999999999999e-06 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851797+0000", 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7969999999999997e-06 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851801+0000", 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6679999999999999e-06 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936554+0000", 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084753203999999999 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936625+0000", 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1406e-05 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936750+0000", 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012416700000000001 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254166 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 6144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.460858801000001, 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.085115273000000005, 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254166 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856445+0000", 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046400929999999996 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856451+0000", 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5119999999999997e-06 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856463+0000", 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2252999999999999e-05 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856470+0000", 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4160000000000001e-06 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936767+0000", 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080297258999999996 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936832+0000", 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4685999999999997e-05 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936920+0000", 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8053999999999994e-05 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.415 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:21.417 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool pool_client.0 2024-09-16T13:01:21.417 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:21.412+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:21.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:21.517 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:21.512+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:21.526 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:21.526 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:21.536 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:21.618 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:21.611+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:21.638 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:21.638 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:21.638 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:21.638 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:21.638 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:21.647 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:21.719 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:21.716+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:21.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:01:21.796 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.796 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260177 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715248510999999, 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039279541000000001, 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260177 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075845+0000", 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075847+0000", 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075852+0000", 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.507e-06 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075856+0000", 2024-09-16T13:01:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6389999999999998e-06 2024-09-16T13:01:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.114965+0000", 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039109496000000001 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115034+0000", 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8378000000000002e-05 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115124+0000", 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0334999999999995e-05 2024-09-16T13:01:21.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260178 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715233114999997, 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039530044, 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260178 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075866+0000", 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0909999999999998e-06 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075872+0000", 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.139e-06 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075876+0000", 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0419999999999996e-06 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115194+0000", 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039317193 2024-09-16T13:01:21.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115259+0000", 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5399000000000002e-05 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115390+0000", 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130803 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260179 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715212264999998, 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039926426000000001, 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260179 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:01:21.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075883+0000", 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6959999999999999e-06 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075889+0000", 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9479999999999996e-06 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075893+0000", 2024-09-16T13:01:21.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7840000000000001e-06 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115419+0000", 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039526517999999997 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115577+0000", 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015783999999999999 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115807+0000", 2024-09-16T13:01:21.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023010399999999999 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260180 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715196517999999, 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040245398000000002, 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260180 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075897+0000", 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075899+0000", 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8169999999999999e-06 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075904+0000", 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-16T13:01:21.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075908+0000", 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5889999999999999e-06 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115837+0000", 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039929193000000002 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115978+0000", 2024-09-16T13:01:21.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014115800000000001 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116142+0000", 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016352900000000001 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260181 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 180224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715181311000002, 2024-09-16T13:01:21.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040482388000000001, 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260181 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075914+0000", 2024-09-16T13:01:21.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6759999999999999e-06 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075919+0000", 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2360000000000004e-06 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075922+0000", 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116171+0000", 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040248702999999997 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116239+0000", 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8391000000000003e-05 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116394+0000", 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000154575 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.819 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260182 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715166600000003, 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042150168000000002, 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260182 2024-09-16T13:01:21.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075927+0000", 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075929+0000", 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7859999999999999e-06 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075934+0000", 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7679999999999997e-06 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075938+0000", 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2590000000000001e-06 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116426+0000", 2024-09-16T13:01:21.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040488077999999997 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116493+0000", 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7681000000000005e-05 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118077+0000", 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015832260000000001 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260183 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715151612, 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040886082999999997, 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260183 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.825 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075942+0000", 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075943+0000", 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.778e-06 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075949+0000", 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-16T13:01:21.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075953+0000", 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116672+0000", 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040719563 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116743+0000", 2024-09-16T13:01:21.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0729000000000007e-05 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116827+0000", 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4364999999999996e-05 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260184 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715136448999999, 2024-09-16T13:01:21.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041043524999999997, 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260184 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075959+0000", 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6869999999999999e-06 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075964+0000", 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.242e-06 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075967+0000", 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116850+0000", 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040882977000000001 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116916+0000", 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5413000000000005e-05 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117000+0000", 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4431999999999994e-05 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.833 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.834 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260185 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.834 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:21.834 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.71512121, 2024-09-16T13:01:21.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041177895999999999, 2024-09-16T13:01:21.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260185 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075974+0000", 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0049999999999999e-06 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075980+0000", 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7470000000000002e-06 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075984+0000", 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117022+0000", 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041038789999999999 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117090+0000", 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7712000000000005e-05 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117150+0000", 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9542e-05 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260257 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:21.838 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715079355999997, 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041222406000000003, 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260257 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076017+0000", 2024-09-16T13:01:21.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4990000000000001e-06 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076024+0000", 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076028+0000", 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9319999999999998e-06 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117162+0000", 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04113472 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117199+0000", 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6507999999999997e-05 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117236+0000", 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7023000000000001e-05 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260258 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715060837000003, 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041343291999999997, 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260258 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:21.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076035+0000", 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.058e-06 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076041+0000", 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.998e-06 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076044+0000", 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3900000000000002e-06 2024-09-16T13:01:21.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117256+0000", 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041212555999999997 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117323+0000", 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6568000000000004e-05 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117375+0000", 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2361999999999998e-05 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260259 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715045418999999, 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041437587999999997, 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260259 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5499999999999999e-07 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076050+0000", 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076055+0000", 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5609999999999998e-06 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076059+0000", 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117396+0000", 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041336879999999999 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117454+0000", 2024-09-16T13:01:21.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8443999999999998e-05 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117485+0000", 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1047999999999998e-05 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260260 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:21.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.71503062, 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041496984000000001, 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260260 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:21.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076063+0000", 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076070+0000", 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7919999999999993e-06 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076076+0000", 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9359999999999996e-06 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076080+0000", 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8240000000000001e-06 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117497+0000", 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041416896000000002 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117531+0000", 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4e-05 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117559+0000", 2024-09-16T13:01:21.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8177000000000001e-05 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260261 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.715008384000001, 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041555793000000001, 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260261 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:21.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076087+0000", 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7850000000000001e-06 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076093+0000", 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6879999999999996e-06 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076096+0000", 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117570+0000", 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041474329999999997 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117611+0000", 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0852999999999997e-05 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117640+0000", 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9213e-05 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.860 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260262 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.714992938000002, 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041643056999999997, 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260262 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076102+0000", 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8509999999999999e-06 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076108+0000", 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8560000000000003e-06 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076111+0000", 2024-09-16T13:01:21.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0620000000000001e-06 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117651+0000", 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041539980999999997 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117703+0000", 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1975000000000003e-05 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117743+0000", 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9872e-05 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260263 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.714977957000002, 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041724412000000002, 2024-09-16T13:01:21.865 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260263 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076117+0000", 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.762e-06 2024-09-16T13:01:21.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076123+0000", 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076126+0000", 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.427e-06 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117757+0000", 2024-09-16T13:01:21.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041631230999999998 2024-09-16T13:01:21.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117792+0000", 2024-09-16T13:01:21.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4078999999999997e-05 2024-09-16T13:01:21.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117839+0000", 2024-09-16T13:01:21.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7861000000000001e-05 2024-09-16T13:01:21.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.871 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.871 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260264 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.871 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:21.871 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.714963111000003, 2024-09-16T13:01:21.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041783638999999997, 2024-09-16T13:01:21.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260264 2024-09-16T13:01:21.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.873 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:21.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:21.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:21.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:01:21.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076132+0000", 2024-09-16T13:01:21.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.751e-06 2024-09-16T13:01:21.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076138+0000", 2024-09-16T13:01:21.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6729999999999997e-06 2024-09-16T13:01:21.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076141+0000", 2024-09-16T13:01:21.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T13:01:21.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117851+0000", 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04171064 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117885+0000", 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3807999999999999e-05 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117913+0000", 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8229999999999999e-05 2024-09-16T13:01:21.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260265 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.714948507999999, 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042095026000000001, 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260265 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076145+0000", 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076147+0000", 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076152+0000", 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5130000000000004e-06 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076156+0000", 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117924+0000", 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041768179000000002 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117980+0000", 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6125000000000002e-05 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118239+0000", 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025934200000000003 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260266 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.883 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.714933639999998, 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045273031999999998, 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260266 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076160+0000", 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076161+0000", 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.796e-06 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076167+0000", 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-06 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076171+0000", 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.579e-06 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.120966+0000", 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.044795297999999997 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121159+0000", 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192618 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121432+0000", 2024-09-16T13:01:21.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000273663 2024-09-16T13:01:21.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.888 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.888 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260267 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:21.888 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:21.888 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.714917262999997, 2024-09-16T13:01:21.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046181535000000003, 2024-09-16T13:01:21.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:21.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:21.889 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:21.889 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:21.889 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:21.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260267 2024-09-16T13:01:21.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.889 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:21.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:21.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076178+0000", 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7969999999999999e-06 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076184+0000", 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6570000000000002e-06 2024-09-16T13:01:21.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076187+0000", 2024-09-16T13:01:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.455e-06 2024-09-16T13:01:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122253+0000", 2024-09-16T13:01:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046065813999999997 2024-09-16T13:01:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122318+0000", 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5338000000000004e-05 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122357+0000", 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9088999999999998e-05 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:21.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:21.894 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:21.895 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:21.895 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:21.896 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:21.896 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:21.887+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:21.916 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:21.940 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:21.975 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:21.995 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:21.987+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:22.009 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:22.043 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:22.043 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:22.047 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:22.047 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:22.050 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:22.050 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:22.061 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:22.105 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:22.099+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:22.197 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:22.187+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:22.210 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:22.210 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:22.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:22.298 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:22.292+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:22.324 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:22.324 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:22.334 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:22.398 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:22.391+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:22.440 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:22.440 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:22.449 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:22.499 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:22.496+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:22.552 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:22.553 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:22.561 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:22.600 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:22.591+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:22.624 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2024-09-16T13:01:22.661 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:22.662 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:22.671 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:22.701 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:22.696+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:22.765 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:22.765 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:22.774 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:22.802 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:22.796+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:22.869 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:22.870 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:22.879 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:22.903 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:22.895+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:22.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T13:01:23.004 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:22.995+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:23.030 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:23.030 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:23.041 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:23.105 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:23.095+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:23.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:23.139 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:23.149 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:23.206 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:23.199+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:23.246 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:23.246 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:23.255 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:23.307 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:23.299+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:23.351 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:23.351 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:23.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:23.408 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:23.404+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:23.469 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:23.469 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:23.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:23.509 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:23.504+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:23.575 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:23.575 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:23.584 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:23.609 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:23.603+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:23.687 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:23.687 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:23.696 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:23.711 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:23.703+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:23.792 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:23.792 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:23.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:23.846 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:23.839+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:23.903 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T13:01:23.912 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:23.908+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:23.961 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:23.961 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:23.971 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:24.013 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:24.008+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:24.048 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:01:24.078 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:24.078 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:24.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:24.137 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:24.131+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:24.184 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:24.184 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:24.184 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:24.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:24.203 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:24.214 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:24.208+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:24.225 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:24.245 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:24.266 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:24.288 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:24.291 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:24.292 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:24.292 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:24.292 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:24.292 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:24.301 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:24.312 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:24.312 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:24.313 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:24.313 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:24.313 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:24.313 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:24.313 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:24.313 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:24.313 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:24.314 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:24.307+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:24.317 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:24.317 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:24.397 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:24.398 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:24.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:24.398 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:24.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.398 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260177 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.398 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:24.398 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316814790000002, 2024-09-16T13:01:24.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039279541000000001, 2024-09-16T13:01:24.398 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260177 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075845+0000", 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075847+0000", 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075852+0000", 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.507e-06 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075856+0000", 2024-09-16T13:01:24.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6389999999999998e-06 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.114965+0000", 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039109496000000001 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115034+0000", 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8378000000000002e-05 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115124+0000", 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0334999999999995e-05 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260178 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316799394, 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039530044, 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260178 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075866+0000", 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0909999999999998e-06 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075872+0000", 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.139e-06 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075876+0000", 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0419999999999996e-06 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115194+0000", 2024-09-16T13:01:24.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039317193 2024-09-16T13:01:24.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115259+0000", 2024-09-16T13:01:24.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5399000000000002e-05 2024-09-16T13:01:24.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115390+0000", 2024-09-16T13:01:24.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130803 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260179 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316778544000002, 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039926426000000001, 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.409 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.409 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.409 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260179 2024-09-16T13:01:24.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.409 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:24.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:24.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075883+0000", 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6959999999999999e-06 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075889+0000", 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9479999999999996e-06 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075893+0000", 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7840000000000001e-06 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115419+0000", 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039526517999999997 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115577+0000", 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015783999999999999 2024-09-16T13:01:24.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115807+0000", 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023010399999999999 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260180 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:24.413 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316762797000003, 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040245398000000002, 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260180 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:24.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075897+0000", 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075899+0000", 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8169999999999999e-06 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075904+0000", 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075908+0000", 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5889999999999999e-06 2024-09-16T13:01:24.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115837+0000", 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039929193000000002 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115978+0000", 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014115800000000001 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116142+0000", 2024-09-16T13:01:24.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016352900000000001 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260181 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 180224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316747589999999, 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040482388000000001, 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.418 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260181 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075914+0000", 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6759999999999999e-06 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075919+0000", 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2360000000000004e-06 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075922+0000", 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116171+0000", 2024-09-16T13:01:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040248702999999997 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116239+0000", 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8391000000000003e-05 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116394+0000", 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000154575 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260182 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316732879, 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042150168000000002, 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260182 2024-09-16T13:01:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075927+0000", 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075929+0000", 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7859999999999999e-06 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075934+0000", 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7679999999999997e-06 2024-09-16T13:01:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075938+0000", 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2590000000000001e-06 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116426+0000", 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040488077999999997 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116493+0000", 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7681000000000005e-05 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118077+0000", 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015832260000000001 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260183 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316717891000003, 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040886082999999997, 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260183 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075942+0000", 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075943+0000", 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.778e-06 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075949+0000", 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075953+0000", 2024-09-16T13:01:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116672+0000", 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040719563 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116743+0000", 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0729000000000007e-05 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116827+0000", 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4364999999999996e-05 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260184 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316702728000003, 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041043524999999997, 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260184 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075959+0000", 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6869999999999999e-06 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075964+0000", 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.242e-06 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075967+0000", 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116850+0000", 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040882977000000001 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116916+0000", 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5413000000000005e-05 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117000+0000", 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4431999999999994e-05 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260185 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316687489000003, 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041177895999999999, 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260185 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075974+0000", 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0049999999999999e-06 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075980+0000", 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7470000000000002e-06 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075984+0000", 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:01:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117022+0000", 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041038789999999999 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117090+0000", 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7712000000000005e-05 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117150+0000", 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9542e-05 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260257 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316645635, 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041222406000000003, 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260257 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076017+0000", 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4990000000000001e-06 2024-09-16T13:01:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076024+0000", 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076028+0000", 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9319999999999998e-06 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117162+0000", 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04113472 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117199+0000", 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6507999999999997e-05 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117236+0000", 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7023000000000001e-05 2024-09-16T13:01:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260258 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316627115999999, 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041343291999999997, 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260258 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076035+0000", 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.058e-06 2024-09-16T13:01:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076041+0000", 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.998e-06 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076044+0000", 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3900000000000002e-06 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117256+0000", 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041212555999999997 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117323+0000", 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6568000000000004e-05 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117375+0000", 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2361999999999998e-05 2024-09-16T13:01:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260259 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316611698000003, 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041437587999999997, 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260259 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5499999999999999e-07 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076050+0000", 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076055+0000", 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5609999999999998e-06 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076059+0000", 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117396+0000", 2024-09-16T13:01:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041336879999999999 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117454+0000", 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8443999999999998e-05 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117485+0000", 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1047999999999998e-05 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260260 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316596898999997, 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041496984000000001, 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260260 2024-09-16T13:01:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076063+0000", 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076070+0000", 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7919999999999993e-06 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076076+0000", 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9359999999999996e-06 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076080+0000", 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8240000000000001e-06 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117497+0000", 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041416896000000002 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117531+0000", 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4e-05 2024-09-16T13:01:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117559+0000", 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8177000000000001e-05 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260261 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316574662999997, 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041555793000000001, 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260261 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076087+0000", 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7850000000000001e-06 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076093+0000", 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6879999999999996e-06 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076096+0000", 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T13:01:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117570+0000", 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041474329999999997 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117611+0000", 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0852999999999997e-05 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117640+0000", 2024-09-16T13:01:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9213e-05 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260262 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316559216999998, 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041643056999999997, 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260262 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076102+0000", 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8509999999999999e-06 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076108+0000", 2024-09-16T13:01:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8560000000000003e-06 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076111+0000", 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0620000000000001e-06 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117651+0000", 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041539980999999997 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117703+0000", 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1975000000000003e-05 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117743+0000", 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9872e-05 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260263 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316544235999999, 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041724412000000002, 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260263 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076117+0000", 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.762e-06 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076123+0000", 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076126+0000", 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.427e-06 2024-09-16T13:01:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117757+0000", 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041631230999999998 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117792+0000", 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4078999999999997e-05 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117839+0000", 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7861000000000001e-05 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260264 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316529389999999, 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041783638999999997, 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260264 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076132+0000", 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.751e-06 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076138+0000", 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6729999999999997e-06 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076141+0000", 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117851+0000", 2024-09-16T13:01:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04171064 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117885+0000", 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3807999999999999e-05 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117913+0000", 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8229999999999999e-05 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260265 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316514787000003, 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042095026000000001, 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260265 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076145+0000", 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:01:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076147+0000", 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076152+0000", 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5130000000000004e-06 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076156+0000", 2024-09-16T13:01:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117924+0000", 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041768179000000002 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117980+0000", 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6125000000000002e-05 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118239+0000", 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025934200000000003 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260266 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316499919000002, 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045273031999999998, 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260266 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076160+0000", 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076161+0000", 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.796e-06 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076167+0000", 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-06 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076171+0000", 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.579e-06 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.120966+0000", 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.044795297999999997 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121159+0000", 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192618 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121432+0000", 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000273663 2024-09-16T13:01:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260267 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.316483542, 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046181535000000003, 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260267 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076178+0000", 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7969999999999999e-06 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076184+0000", 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6570000000000002e-06 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076187+0000", 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.455e-06 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122253+0000", 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046065813999999997 2024-09-16T13:01:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122318+0000", 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5338000000000004e-05 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122357+0000", 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9088999999999998e-05 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.493 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:24.494 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:24.495 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:24.488+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:24.595 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:24.587+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:24.601 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:24.601 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:24.601 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:24.611 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:24.696 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:24.687+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:24.713 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:24.714 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:24.714 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:24.714 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:24.714 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:24.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:24.797 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:24.787+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.750277892, 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:01:24.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:01:24.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:01:24.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.750225473, 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.828 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:24.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:24.832 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.750208624999999, 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:01:24.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.836 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.750192511000002, 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.750127581000001, 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:01:24.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:01:24.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.750111482000001, 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:24.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.750037700999997, 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:01:24.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:01:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:01:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:01:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.750020499999998, 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.750003796999998, 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.860 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:01:24.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:01:24.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.213692541999997, 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.864 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.213675406, 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.213655824, 2024-09-16T13:01:24.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:01:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:01:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:01:24.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.877 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.213624965000001, 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:24.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:01:24.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:01:24.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.213556869000001, 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:24.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:01:24.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.886 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.206253013999998, 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.206235425000003, 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:01:24.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.896 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.206220532000003, 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.206188914999998, 2024-09-16T13:01:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:01:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:01:24.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:01:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:01:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:01:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:01:24.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.206170894000003, 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.292135521999999, 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2569454119999999, 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:24.910 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 279128 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527093+0000", 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4040000000000002e-06 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:24.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527104+0000", 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-05 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527133+0000", 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8909999999999999e-05 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527144+0000", 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1497e-05 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527223+0000", 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8577000000000003e-05 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527236+0000", 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3362e-05 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783387+0000", 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2561508619999999 2024-09-16T13:01:24.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783699+0000", 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311687 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.784036+0000", 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033771300000000001 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:24.917 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:24.917 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:24.918 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:24.919 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:24.911+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:25.019 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:25.016+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:25.038 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:25.038 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:25.038 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:25.048 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:25.120 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:25.116+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:25.145 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:25.145 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:25.154 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:25.221 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:25.215+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:25.253 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:25.253 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:25.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:25.253 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:25.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.253 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240664 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 152576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.253 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:25.253 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.705505791, 2024-09-16T13:01:25.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047790817999999999, 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240664 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1900000000000003e-07 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543064+0000", 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4449999999999999e-06 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543079+0000", 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5221e-05 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543084+0000", 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8590000000000004e-06 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590750+0000", 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047665710999999999 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590801+0000", 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1827999999999999e-05 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590852+0000", 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0235000000000003e-05 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240665 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.258 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.705476388000001, 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047896192999999997, 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240665 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:25.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543091+0000", 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543094+0000", 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7149999999999998e-06 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543102+0000", 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6549999999999993e-06 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543108+0000", 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4840000000000003e-06 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590885+0000", 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047777396 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590936+0000", 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0611e-05 2024-09-16T13:01:25.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590987+0000", 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0785000000000003e-05 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240666 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 162816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:25.263 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.705454119999999, 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048010069000000002, 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240666 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:25.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:25.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8500000000000002e-07 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543116+0000", 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.531e-06 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543123+0000", 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1069999999999999e-06 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543127+0000", 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4309999999999996e-06 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591020+0000", 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047892413000000002 2024-09-16T13:01:25.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591071+0000", 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1650000000000002e-05 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591123+0000", 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1452000000000001e-05 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240667 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.705435338000001, 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048125951, 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240667 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.269 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543132+0000", 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5099999999999999e-07 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543134+0000", 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1500000000000002e-06 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543141+0000", 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543146+0000", 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9200000000000003e-06 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591158+0000", 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048011246 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591207+0000", 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9295999999999997e-05 2024-09-16T13:01:25.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591257+0000", 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0426000000000001e-05 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240668 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 173056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.705415633000001, 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048243280999999999, 2024-09-16T13:01:25.273 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240668 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543152+0000", 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543154+0000", 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5380000000000001e-06 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543161+0000", 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7920000000000004e-06 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543172+0000", 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1327e-05 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591290+0000", 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048118181000000003 2024-09-16T13:01:25.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591344+0000", 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4061000000000003e-05 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591394+0000", 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9866e-05 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240766 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.704211307999998, 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047716644000000003, 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240766 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544356+0000", 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2500000000000005e-07 2024-09-16T13:01:25.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544373+0000", 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6685999999999999e-05 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544381+0000", 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7190000000000001e-06 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544384+0000", 2024-09-16T13:01:25.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591969+0000", 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047585483999999997 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592021+0000", 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1839000000000003e-05 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592072+0000", 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754000000000002e-05 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240767 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 201728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.704179433, 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047822479000000001, 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240767 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544388+0000", 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.51e-07 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544390+0000", 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.824e-06 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544393+0000", 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.681e-06 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544396+0000", 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4940000000000002e-06 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592105+0000", 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047709184000000002 2024-09-16T13:01:25.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592158+0000", 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2772999999999998e-05 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592210+0000", 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2071999999999999e-05 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240768 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.704166632000003, 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047944647, 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240768 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0400000000000002e-07 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544403+0000", 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1430000000000001e-06 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544406+0000", 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7450000000000001e-06 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544409+0000", 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592241+0000", 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047832146999999998 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592291+0000", 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9289000000000002e-05 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592345+0000", 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4067000000000003e-05 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240769 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.704155006999997, 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048066225999999997, 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.291 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240769 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1399999999999998e-07 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544413+0000", 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.42e-06 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544417+0000", 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.529e-06 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544420+0000", 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5620000000000002e-06 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592378+0000", 2024-09-16T13:01:25.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047958354000000002 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592427+0000", 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9013e-05 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592478+0000", 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1034e-05 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.295 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240770 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.704144743999997, 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048189432999999997, 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.296 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240770 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3200000000000001e-07 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544424+0000", 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4449999999999999e-06 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544427+0000", 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5049999999999999e-06 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544430+0000", 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.61e-06 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592511+0000", 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048080912000000003 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592563+0000", 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2107000000000001e-05 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592611+0000", 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8522000000000001e-05 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.300 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240771 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.704110137000001, 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290128000000002, 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240771 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5700000000000002e-07 2024-09-16T13:01:25.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544459+0000", 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6840000000000001e-06 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544466+0000", 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999997e-06 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544469+0000", 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592645+0000", 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048175216999999999 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592696+0000", 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1292999999999998e-05 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592747+0000", 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0751000000000002e-05 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240772 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.704086842000002, 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048403333999999999, 2024-09-16T13:01:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240772 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544482+0000", 2024-09-16T13:01:25.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8390000000000001e-06 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544486+0000", 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9369999999999997e-06 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544489+0000", 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8459999999999999e-06 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592779+0000", 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290301000000001 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592831+0000", 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2335e-05 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592883+0000", 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1786999999999997e-05 2024-09-16T13:01:25.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240773 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.704075457000002, 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048527963, 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.310 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240773 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544492+0000", 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.84e-07 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544493+0000", 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6279999999999999e-06 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544498+0000", 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4689999999999999e-06 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544500+0000", 2024-09-16T13:01:25.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2024-09-16T13:01:25.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592916+0000", 2024-09-16T13:01:25.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048415672 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592965+0000", 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9120999999999999e-05 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593019+0000", 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3950999999999999e-05 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240774 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.704063802, 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048649546000000002, 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240774 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:01:25.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544505+0000", 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.61e-06 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544508+0000", 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4929999999999998e-06 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544511+0000", 2024-09-16T13:01:25.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4169999999999999e-06 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593051+0000", 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048539604 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593103+0000", 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2068999999999999e-05 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593152+0000", 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9870999999999997e-05 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240775 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.704053547999997, 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048773328999999997, 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.319 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240775 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545618+0000", 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011045810000000001 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545625+0000", 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6079999999999998e-06 2024-09-16T13:01:25.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545644+0000", 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.823e-05 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545653+0000", 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3000000000000007e-06 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593186+0000", 2024-09-16T13:01:25.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047533387000000003 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593235+0000", 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8767000000000003e-05 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593286+0000", 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1456000000000003e-05 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240776 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.702901410000003, 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047754889000000002, 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240776 2024-09-16T13:01:25.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545666+0000", 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-06 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545671+0000", 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2270000000000004e-06 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545681+0000", 2024-09-16T13:01:25.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0506000000000001e-05 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545687+0000", 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0970000000000002e-06 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593320+0000", 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047632999000000002 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593369+0000", 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8882999999999999e-05 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593420+0000", 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1127e-05 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240777 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:25.328 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.702872788999997, 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047878318000000003, 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240777 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545695+0000", 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545698+0000", 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0709999999999999e-06 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545707+0000", 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7629999999999995e-06 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545713+0000", 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2839999999999997e-06 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593453+0000", 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047740281000000002 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593518+0000", 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5493000000000005e-05 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593572+0000", 2024-09-16T13:01:25.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3739000000000002e-05 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240778 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.702848349999996, 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047999411999999998, 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.333 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240778 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545719+0000", 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.13e-07 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545728+0000", 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1560000000000001e-06 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545901+0000", 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017311599999999999 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545909+0000", 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.028e-06 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593606+0000", 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047696660000000002 2024-09-16T13:01:25.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593666+0000", 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9611999999999998e-05 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593718+0000", 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2227e-05 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240951 4.8 4:10cc0d5f:::rbd_data.11b552946c0f.0000000000000120:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.694768080999999, 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048226894999999999, 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.338 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240951 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2300000000000002e-07 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553801+0000", 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.686e-06 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553805+0000", 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3039999999999998e-06 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553809+0000", 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.599994+0000", 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046185154999999999 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.600112+0000", 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118589 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.602026+0000", 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019131370000000001 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:272626 4.12 4:4935e950:::rbd_data.11b59088eb24.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.790740458999998, 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022439990000001, 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 272626 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.343 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457828+0000", 2024-09-16T13:01:25.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457835+0000", 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7900000000000002e-06 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457853+0000", 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7671000000000001e-05 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457859+0000", 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2949999999999999e-06 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457913+0000", 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4112e-05 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457922+0000", 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0850000000000003e-06 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.457879+0000", 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.99995641700000004 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458079+0000", 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020072300000000001 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458304+0000", 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000224601 2024-09-16T13:01:25.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459393+0000", 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010886170000000001 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459877+0000", 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048443500000000002 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460006+0000", 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012914 2024-09-16T13:01:25.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460070+0000", 2024-09-16T13:01:25.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4074999999999999e-05 2024-09-16T13:01:25.349 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.349 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.349 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.349 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.349 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.349 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:25.351 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:25.351 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:25.343+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:25.452 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:25.443+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:25.459 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:25.459 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:25.459 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:25.459 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:25.468 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:25.553 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:25.543+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:25.569 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:25.569 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:25.569 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:25.569 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:25.569 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:25.578 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:25.653 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:25.647+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:25.681 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:25.681 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 160.12666368699999, 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:01:25.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:01:25.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:01:25.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:01:25.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.30154664, 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.688 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:01:25.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:01:25.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:01:25.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:25.693 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.30149938400001, 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:01:25.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:01:25.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.30129644799999, 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:01:25.699 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:01:25.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:01:25.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:01:25.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.30126518599999, 2024-09-16T13:01:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:01:25.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:01:25.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.30122942400001, 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.711 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:01:25.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.301034794, 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.717 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:01:25.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:01:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:01:25.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:25.722 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.30098794899999, 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:25.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:01:25.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:01:25.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 158.75671177800001, 2024-09-16T13:01:25.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:01:25.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 158.75665787099999, 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.734 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:01:25.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:01:25.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:01:25.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 158.75661556599999, 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.740 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:25.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:01:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 158.756457141, 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.746 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:25.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:01:25.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 158.75640766000001, 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.752 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:01:25.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:01:25.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 158.75627709, 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.758 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:25.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:01:25.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 158.756095577, 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.764 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:01:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 157.94556790999999, 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:25.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:01:25.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 157.945418956, 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:25.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:01:25.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64507 4.1e e83/31 4:7f7c7834:::rbd_data.11b5c2ee3542.0000000000000008:head v 83'225, mlcod=81'223)", 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 66.906005668999995, 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021284401000000001, 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.782 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770380+0000", 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8999999999999998e-07 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777141+0000", 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067617670000000001 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777157+0000", 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777167+0000", 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777267+0000", 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100364 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777304+0000", 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7165000000000001e-05 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791601+0000", 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014297275999999999 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791664+0000", 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2217000000000004e-05 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.786 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64509 4.15 e83/31 4:ad2e18f4:::rbd_data.11b5c2ee3542.0000000000000009:head v 83'183, mlcod=81'180)", 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 66.899247767000006, 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014054395000000001, 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777138+0000", 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778534+0000", 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013956769999999999 2024-09-16T13:01:25.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778555+0000", 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1046000000000001e-05 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778564+0000", 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9290000000000004e-06 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778633+0000", 2024-09-16T13:01:25.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9363000000000007e-05 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778666+0000", 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135000000000003e-05 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791145+0000", 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012479743 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791192+0000", 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6156000000000003e-05 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:217754 4.17 e92/31 4:ee4048fa:::rbd_data.11b58362ce83.0000000000000002:head v 92'697, mlcod=91'693)", 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.177077713000003, 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011416822, 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:25.791 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499309+0000", 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6929999999999999e-06 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499322+0000", 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499354+0000", 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135999999999999e-05 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499365+0000", 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111e-05 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499504+0000", 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013877599999999999 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499565+0000", 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1939000000000004e-05 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510591+0000", 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011025874999999999 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510724+0000", 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013266299999999999 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:25.795 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:25.797 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:25.797 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:25.792+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:25.897 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:25.891+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:25.915 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T13:01:25.955 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:25.956 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:25.956 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:25.956 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:25.966 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:25.998 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:25.991+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:26.065 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:26.065 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:26.065 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:26.065 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:26.065 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:26.075 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:26.099 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:26.091+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:26.172 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:26.172 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:26.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:26.172 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:26.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.172 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-16T13:01:26.172 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:26.172 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.632045477999998, 2024-09-16T13:01:26.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98766548899999995, 2024-09-16T13:01:26.172 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279128 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534648+0000", 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1459999999999999e-06 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534655+0000", 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5220000000000004e-06 2024-09-16T13:01:26.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534674+0000", 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9717000000000001e-05 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534683+0000", 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534759+0000", 2024-09-16T13:01:26.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6074000000000004e-05 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534770+0000", 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0665999999999999e-05 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522281+0000", 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98751079600000002 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522312+0000", 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0876000000000001e-05 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.9 109/108 [PushOp(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29, version: 106'278, data_included: [0~688128,802816~10077,884736~295,1950271~32193], data_size: 730693, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29@106'278, size: 1982464, copy_subset: [0~1982464], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1982464, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.525026346000001, 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067875690000000002, 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.177 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641666+0000", 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3060000000000001e-06 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644856+0000", 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00318995 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644894+0000", 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7382999999999998e-05 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644913+0000", 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9415999999999998e-05 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645084+0000", 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017036899999999999 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645106+0000", 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2887e-05 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.648453+0000", 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003346258 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280527 4.13 4:cc227b3c:::rbd_data.11b5cb8797c6.0000000000000000:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.293566988999999, 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052047300000000003, 2024-09-16T13:01:26.181 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280527 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:26.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873126+0000", 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6530000000000001e-06 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873133+0000", 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0940000000000002e-06 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873153+0000", 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9538000000000002e-05 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873161+0000", 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1009999999999992e-06 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873245+0000", 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4075999999999999e-05 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873360+0000", 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114951 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873530+0000", 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016994699999999999 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.877617+0000", 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040873000000000003 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878160+0000", 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054299199999999995 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878251+0000", 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1062000000000003e-05 2024-09-16T13:01:26.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878329+0000", 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8016000000000007e-05 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280530 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.293504984999998, 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056839330000000004, 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280530 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:26.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8199999999999999e-07 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873191+0000", 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4599999999999999e-06 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873198+0000", 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8399999999999997e-06 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873201+0000", 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.641e-06 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873298+0000", 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6682999999999996e-05 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873451+0000", 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015304000000000001 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873692+0000", 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024124300000000001 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.876380+0000", 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026873980000000001 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878771+0000", 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002391013 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878824+0000", 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3136000000000001e-05 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878870+0000", 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6796999999999999e-05 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.192 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280650 4.1b 4:d8560506:::rbd_header.11b5de89dd5b:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.407180575000002, 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085055370000000005, 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280650 2024-09-16T13:01:26.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759514+0000", 2024-09-16T13:01:26.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7429999999999998e-06 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759524+0000", 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8309999999999996e-06 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759540+0000", 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6506000000000001e-05 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759548+0000", 2024-09-16T13:01:26.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1810000000000001e-06 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.761995+0000", 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002446518 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.762120+0000", 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125124 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.762945+0000", 2024-09-16T13:01:26.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082531599999999998 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.766013+0000", 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030673549999999999 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.767864+0000", 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001851161 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.767939+0000", 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5012000000000006e-05 2024-09-16T13:01:26.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.768016+0000", 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7789999999999999e-05 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280740 4.18 4:1df486ae:::rbd_data.11b5de89dd5b.0000000000000000:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.257688446, 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070806940000000002, 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.198 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280740 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909005+0000", 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.317e-06 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909016+0000", 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0852999999999999e-05 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909052+0000", 2024-09-16T13:01:26.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5256000000000003e-05 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909064+0000", 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2413e-05 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909168+0000", 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103615 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909335+0000", 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016715100000000001 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909599+0000", 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000264391 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.914767+0000", 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051679309999999997 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915839+0000", 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001072415 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915986+0000", 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014609099999999999 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.916084+0000", 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8261e-05 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.203 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280741 4.13 4:ce26f199:::rbd_data.11b5de89dd5b.0000000000000001:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.257601858000001, 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062230289999999997, 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280741 2024-09-16T13:01:26.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909091+0000", 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0100000000000001e-06 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909096+0000", 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4399999999999996e-06 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909107+0000", 2024-09-16T13:01:26.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.066e-05 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909113+0000", 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9460000000000003e-06 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909206+0000", 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3450999999999997e-05 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909341+0000", 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135267 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909519+0000", 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177603 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.913993+0000", 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044734989999999997 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915108+0000", 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001115001 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915207+0000", 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9562999999999995e-05 2024-09-16T13:01:26.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915313+0000", 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010558899999999999 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281211 4.5 4:a14f106d:::rbd_header.11b515823229:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:26.210 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.369054845000001, 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093462500000000004, 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281211 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797641+0000", 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4259999999999996e-06 2024-09-16T13:01:26.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797662+0000", 2024-09-16T13:01:26.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.069e-05 2024-09-16T13:01:26.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797705+0000", 2024-09-16T13:01:26.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3495999999999998e-05 2024-09-16T13:01:26.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797719+0000", 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4003e-05 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797828+0000", 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010857 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797840+0000", 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.235e-05 2024-09-16T13:01:26.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.802951+0000", 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051111079999999996 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.803147+0000", 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000196109 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806806+0000", 2024-09-16T13:01:26.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003658246 2024-09-16T13:01:26.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806865+0000", 2024-09-16T13:01:26.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9246999999999997e-05 2024-09-16T13:01:26.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806983+0000", 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011800500000000001 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281308 4.1b 4:db2b1562:::rbd_data.11b515823229.000000000000002f:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.242155834, 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056950619999999999, 2024-09-16T13:01:26.218 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281308 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7999999999999995e-07 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924539+0000", 2024-09-16T13:01:26.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9799999999999998e-06 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924546+0000", 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4620000000000001e-06 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924550+0000", 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0040000000000001e-06 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924628+0000", 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7749000000000004e-05 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924853+0000", 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000225491 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.925042+0000", 2024-09-16T13:01:26.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018871099999999999 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928196+0000", 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031538120000000002 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930092+0000", 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001896556 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930167+0000", 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5111999999999995e-05 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930231+0000", 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3504999999999996e-05 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.224 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281318 4.e 4:7158d438:::rbd_data.11b515823229.0000000000000081:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.237269934, 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058318850000000002, 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281318 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929423+0000", 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2139999999999999e-06 2024-09-16T13:01:26.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929428+0000", 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.826e-06 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929439+0000", 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1549000000000001e-05 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929444+0000", 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0050000000000004e-06 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929506+0000", 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1927999999999994e-05 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929632+0000", 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012569299999999999 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935090+0000", 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054580920000000003 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935154+0000", 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3688000000000004e-05 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935253+0000", 2024-09-16T13:01:26.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9889999999999994e-05 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282172 4.13 4:c99eeb3d:::rbd_data.11b5b3dc7343.0000000000000001:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.942314+0000", 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.224377745, 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005440907, 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282172 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942314+0000", 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942314+0000", 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942316+0000", 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.176e-06 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942326+0000", 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0514e-05 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942361+0000", 2024-09-16T13:01:26.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4875000000000001e-05 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942374+0000", 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2924e-05 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942515+0000", 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140746 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.943277+0000", 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076154200000000003 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947497+0000", 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042207939999999999 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947593+0000", 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5166000000000003e-05 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947755+0000", 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016217 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282176 4.1 4:82eb3962:::rbd_data.11b5b3dc7343.0000000000000005:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.942450+0000", 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.224241648, 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051970929999999999, 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.235 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282176 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942450+0000", 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942450+0000", 2024-09-16T13:01:26.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942451+0000", 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1230000000000001e-06 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942457+0000", 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8370000000000002e-06 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942474+0000", 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6663e-05 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942484+0000", 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0462000000000001e-05 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942618+0000", 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134167 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.943112+0000", 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049358500000000005 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.943411+0000", 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000299296 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947145+0000", 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037339589999999998 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947437+0000", 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029157299999999999 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947547+0000", 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110287 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947647+0000", 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100141 2024-09-16T13:01:26.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282181 4.d 4:b2f030fd:::rbd_data.11b5b3dc7343.0000000000000034:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.946061+0000", 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.220630864, 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066091789999999997, 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282181 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946061+0000", 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946061+0000", 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946064+0000", 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0359999999999998e-06 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946076+0000", 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2614e-05 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946100+0000", 2024-09-16T13:01:26.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3198999999999999e-05 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946115+0000", 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5953000000000001e-05 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946312+0000", 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019610499999999999 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946957+0000", 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00064557500000000001 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947239+0000", 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028224599999999999 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.949960+0000", 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027203399999999999 2024-09-16T13:01:26.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952517+0000", 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025567709999999998 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952601+0000", 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4557999999999999e-05 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952670+0000", 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8782000000000007e-05 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282774 4.c 4:36b2d631:::rbd_object_map.11b51a3eb68b:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:19.869669+0000", 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.2970227530000003, 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055610959999999997, 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.247 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282774 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869669+0000", 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869669+0000", 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869671+0000", 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6240000000000002e-06 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869684+0000", 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2152e-05 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869715+0000", 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1659000000000003e-05 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869729+0000", 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3563e-05 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869823+0000", 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4333999999999995e-05 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869831+0000", 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5859999999999999e-06 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.872858+0000", 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030270700000000002 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.873002+0000", 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014394599999999999 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.873214+0000", 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021210200000000001 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875001+0000", 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001786852 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875103+0000", 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102058 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875170+0000", 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7019000000000003e-05 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875230+0000", 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0130999999999997e-05 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:283012 4.9 4:9572e2af:::rbd_data.11b5c2c64266.0000000000000005:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:20.979407+0000", 2024-09-16T13:01:26.253 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1872847350000004, 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070250620000000003, 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 283012 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979407+0000", 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979407+0000", 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979408+0000", 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.514e-06 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979416+0000", 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2830000000000004e-06 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979438+0000", 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2218999999999999e-05 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979450+0000", 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2303000000000001e-05 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979580+0000", 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013011699999999999 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.980107+0000", 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052712500000000003 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.980398+0000", 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029023299999999998 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.984244+0000", 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038468450000000002 2024-09-16T13:01:26.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.986286+0000", 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020413279999999998 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.986384+0000", 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8482999999999999e-05 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.986432+0000", 2024-09-16T13:01:26.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7611999999999997e-05 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:283597 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138)", 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:24.930684+0000", 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2360070750000001, 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013228014, 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 283597 2024-09-16T13:01:26.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.930684+0000", 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.930684+0000", 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.930688+0000", 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3390000000000001e-06 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.930702+0000", 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4637000000000001e-05 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.943493+0000", 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012790625 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.943521+0000", 2024-09-16T13:01:26.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7787e-05 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.943608+0000", 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7653999999999998e-05 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.943809+0000", 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000200991 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.943912+0000", 2024-09-16T13:01:26.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102981 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:283673 4.1b 4:df702a73:::rbd_data.11b52f130126.000000000000004b:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:25.045415+0000", 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.121276559, 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050820750000000001, 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.263 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 283673 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045415+0000", 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045415+0000", 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045418+0000", 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3900000000000002e-06 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045433+0000", 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4253000000000001e-05 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045497+0000", 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4775000000000002e-05 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045518+0000", 2024-09-16T13:01:26.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0553000000000001e-05 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045701+0000", 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018275299999999999 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.046191+0000", 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049032800000000005 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.046488+0000", 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029667000000000002 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.047940+0000", 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014519940000000001 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.050423+0000", 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024837209999999999 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.050460+0000", 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6411000000000001e-05 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.050497+0000", 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7227000000000001e-05 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:283675 4.1b 4:db8634b6:::rbd_data.11b52f130126.0000000000000050:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-16T13:01:26.268 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:25.045592+0000", 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1210997700000001, 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050885870000000003, 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 283675 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045592+0000", 2024-09-16T13:01:26.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045592+0000", 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045593+0000", 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5099999999999999e-06 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045602+0000", 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6470000000000004e-06 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045654+0000", 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2337999999999999e-05 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045667+0000", 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.296e-05 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.047407+0000", 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001740187 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.047566+0000", 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015870599999999999 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.047631+0000", 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4560999999999994e-05 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.048235+0000", 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060450599999999999 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.050613+0000", 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023781129999999998 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.050656+0000", 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2582e-05 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.050680+0000", 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4477e-05 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:283677 4.1a 4:5c9f4a29:::rbd_data.11b52f130126.0000000000000090:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:25.045692+0000", 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1209998619999999, 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004975691, 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 283677 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045692+0000", 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045692+0000", 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045694+0000", 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8700000000000001e-06 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045701+0000", 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6590000000000007e-06 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045755+0000", 2024-09-16T13:01:26.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4116000000000002e-05 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045781+0000", 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5519000000000001e-05 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045876+0000", 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5530000000000002e-05 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045929+0000", 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2981e-05 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.045997+0000", 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7267999999999994e-05 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.047243+0000", 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001246671 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:26.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.050472+0000", 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032284330000000002 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.050580+0000", 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010861699999999999 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.050667+0000", 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7027000000000006e-05 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:283779 4.13 4:ccb453e7:::rbd_header.11b55c5d4be2:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:25.909652+0000", 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.25703974299999999, 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070661450000000002, 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 283779 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909652+0000", 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909652+0000", 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909654+0000", 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2730000000000001e-06 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909665+0000", 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0478e-05 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909690+0000", 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5074000000000001e-05 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909698+0000", 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8130000000000002e-06 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909812+0000", 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011328999999999999 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909824+0000", 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2573e-05 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.914045+0000", 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042204269999999997 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:26.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.914291+0000", 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024655100000000002 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.916385+0000", 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002093953 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.916514+0000", 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012908400000000001 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:26.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:26.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.916718+0000", 2024-09-16T13:01:26.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020362900000000001 2024-09-16T13:01:26.285 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.285 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.285 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.285 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:26.285 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:26.285 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:26.286 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:26.284+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:26.287 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:26.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:01:26.387 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:26.384+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:26.410 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:26.410 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:26.410 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:26.411 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:26.419 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:26.485 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:26.488 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:26.484+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:26.520 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:26.551 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:26.551 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:26.553 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:26.566 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:26.576 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:26.589 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:26.584+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:26.598 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:26.620 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:26.643 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:26.643 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:26.646 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:26.646 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:26.667 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:26.667 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:26.677 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:26.689 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:26.684+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:26.790 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:26.783+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:26.829 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:26.829 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:26.838 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:26.891 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:26.883+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:26.925 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T13:01:26.946 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:26.946 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:26.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:27.001 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:26.995+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:27.091 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:27.083+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:27.108 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:27.108 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:27.118 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:27.193 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:27.187+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:27.214 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:27.214 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:27.223 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:27.294 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:27.287+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:27.371 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:27.371 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:27.380 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:27.425 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:27.419+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:27.482 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:27.482 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:27.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:27.495 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:27.487+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:27.593 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:27.593 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:27.596 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:27.587+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:27.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:27.697 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:27.692+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:27.747 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:27.747 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:27.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:27.798 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:27.791+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:27.857 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:27.857 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:27.866 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:27.909 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:27.903+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:27.980 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:27.980 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:27.989 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:28.000 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:27.991+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:28.089 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:28.089 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:28.099 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:28.101 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:28.091+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:28.193 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:28.194 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:28.197 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 5, 7, 4] 2024-09-16T13:01:28.197 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2024-09-16T13:01:28.197 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T13:01:28.197 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T13:01:28.199 INFO:tasks.ceph.osd.6:Started 2024-09-16T13:01:28.200 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:01:28.201 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:28.195+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:28.203 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:28.302 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:28.302 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:28.302 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:28.295+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:28.303 INFO:teuthology.orchestra.run.smithi136.stderr:no valid command found; 10 closest matches: 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:0 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:1 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:2 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:abort 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:assert 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:config diff 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:config diff get 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:config get 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:config help [] 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:config set ... 2024-09-16T13:01:28.304 INFO:teuthology.orchestra.run.smithi136.stderr:admin_socket: invalid command 2024-09-16T13:01:28.306 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T13:01:28.307 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-16T13:01:28.311 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:28.404 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:28.395+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:28.404 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:28.404 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:28.413 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:28.505 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:28.500+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:28.512 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:28.512 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:28.521 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:28.606 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:28.599+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:28.625 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:28.625 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:28.634 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:28.647 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:01:28.707 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:28.704+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:28.732 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:28.732 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:28.741 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:28.797 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:28.808 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:28.804+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:28.808 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:28.804+0000 7f90445c1540 -1 Falling back to public interface 2024-09-16T13:01:28.820 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:28.840 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:28.840 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:28.842 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:28.849 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:28.860 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:28.902 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:28.908 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:28.899+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:28.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:28.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:28.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:28.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:28.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:28.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:28.925 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:28.925 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:28.929 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:28.929 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:28.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T13:01:28.974 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:28.974 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:28.974 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:28.987 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:29.033 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:29.027+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:29.110 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:29.103+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:29.133 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:29.133 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:29.143 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:29.211 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:29.208+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:29.238 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:29.238 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:29.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:29.238 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:29.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.238 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54690 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1101824~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.238 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:29.238 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540574039999996, 2024-09-16T13:01:29.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055984602000000001, 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54690 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:29.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692992+0000", 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692993+0000", 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-07 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.692994+0000", 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7519999999999999e-06 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693000+0000", 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3990000000000003e-06 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693003+0000", 2024-09-16T13:01:29.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6550000000000002e-06 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748863+0000", 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055859714999999997 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748915+0000", 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1786000000000002e-05 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.748977+0000", 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1922999999999996e-05 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54691 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1134592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540559013999996, 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056124259000000003, 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54691 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693007+0000", 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693008+0000", 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693010+0000", 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1210000000000001e-06 2024-09-16T13:01:29.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693015+0000", 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5879999999999997e-06 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693019+0000", 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8449999999999996e-06 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749015+0000", 2024-09-16T13:01:29.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055995925000000002 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749071+0000", 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5791000000000002e-05 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749131+0000", 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0568e-05 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54692 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1167360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540542991999999, 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056291869000000001, 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54692 2024-09-16T13:01:29.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693023+0000", 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693024+0000", 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693026+0000", 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0190000000000001e-06 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693032+0000", 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-06 2024-09-16T13:01:29.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693035+0000", 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8519999999999997e-06 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749153+0000", 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056117475999999999 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749199+0000", 2024-09-16T13:01:29.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5846000000000001e-05 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749315+0000", 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116421 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54693 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1200128~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:29.252 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540526896000003, 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056410120000000001, 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54693 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:29.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693039+0000", 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693040+0000", 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000004e-07 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693042+0000", 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693047+0000", 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5149999999999997e-06 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693051+0000", 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.053e-06 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749331+0000", 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056279296999999999 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749390+0000", 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9029e-05 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749449+0000", 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9898000000000001e-05 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54694 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1232896~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540511006000003, 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056660407000000003, 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54694 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693055+0000", 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693056+0000", 2024-09-16T13:01:29.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693057+0000", 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.719e-06 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693063+0000", 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8810000000000001e-06 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693067+0000", 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7910000000000002e-06 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749491+0000", 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056424201 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749590+0000", 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8356999999999994e-05 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749716+0000", 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126107 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54695 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1265664~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540495336000006, 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05677546, 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54695 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693071+0000", 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693073+0000", 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.764e-06 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693079+0000", 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8030000000000002e-06 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693083+0000", 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8650000000000003e-06 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749737+0000", 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056653958999999997 2024-09-16T13:01:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749790+0000", 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3640000000000001e-05 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749846+0000", 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6014999999999998e-05 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54696 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1298432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540479589, 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057030674000000003, 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54696 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.266 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693087+0000", 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693089+0000", 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9290000000000001e-06 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693095+0000", 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8889999999999999e-06 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693098+0000", 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4570000000000001e-06 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749895+0000", 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056797179000000003 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.749989+0000", 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3727999999999994e-05 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750117+0000", 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000128147 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54697 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1331200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.270 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540463801000001, 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057202966000000001, 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54697 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:29.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693102+0000", 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693103+0000", 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693105+0000", 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0549999999999998e-06 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693111+0000", 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693114+0000", 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4819999999999999e-06 2024-09-16T13:01:29.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750191+0000", 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057076535999999997 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750240+0000", 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999999e-05 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750305+0000", 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5161999999999997e-05 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54698 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 1363968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540448479000005, 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057410058999999999, 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54698 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:29.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693118+0000", 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693124+0000", 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0249999999999999e-06 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693130+0000", 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8719999999999999e-06 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693134+0000", 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6799999999999999e-06 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750346+0000", 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057211953000000003 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750434+0000", 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8288000000000006e-05 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750528+0000", 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3924000000000004e-05 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.279 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54785 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540428554000002, 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057649537000000001, 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54785 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693138+0000", 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:01:29.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693140+0000", 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693146+0000", 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2580000000000001e-06 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693150+0000", 2024-09-16T13:01:29.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.286e-06 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750573+0000", 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057423167999999997 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750667+0000", 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4215000000000005e-05 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750787+0000", 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012016899999999999 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54786 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 57344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540412606000004, 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057804925, 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54786 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693154+0000", 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-07 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693156+0000", 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693162+0000", 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693165+0000", 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3840000000000001e-06 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750821+0000", 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057656178000000002 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750885+0000", 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3748e-05 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750959+0000", 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3714000000000006e-05 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54787 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 90112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540397444999996, 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058040402999999997, 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54787 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693169+0000", 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:29.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693171+0000", 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0020000000000001e-06 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693177+0000", 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7819999999999999e-06 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693180+0000", 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5130000000000001e-06 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.750998+0000", 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057817325000000003 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751088+0000", 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0647000000000001e-05 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751209+0000", 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120784 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54788 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 122880~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540381796999995, 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058170082999999997, 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54788 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693184+0000", 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693185+0000", 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693187+0000", 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.792e-06 2024-09-16T13:01:29.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693192+0000", 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6389999999999997e-06 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693196+0000", 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5760000000000002e-06 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751232+0000", 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058035984999999998 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751289+0000", 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7614000000000001e-05 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751355+0000", 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5143999999999998e-05 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.297 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54789 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 155648~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540366700000007, 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423719999999998, 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.298 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54789 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693200+0000", 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693202+0000", 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693207+0000", 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5180000000000004e-06 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693211+0000", 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7780000000000001e-06 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751387+0000", 2024-09-16T13:01:29.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058175444 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751503+0000", 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116723 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751623+0000", 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119929 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54790 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 188416~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540351438000002, 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058577514999999997, 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54790 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.303 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693215+0000", 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693217+0000", 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.968e-06 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693223+0000", 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7169999999999996e-06 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693226+0000", 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751669+0000", 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058442475000000001 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751719+0000", 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9889000000000003e-05 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751792+0000", 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3506999999999999e-05 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54791 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 221184~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.307 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540335691999999, 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058799097000000002, 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54791 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693231+0000", 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4799999999999999e-07 2024-09-16T13:01:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693233+0000", 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6789999999999999e-06 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693239+0000", 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0340000000000002e-06 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693242+0000", 2024-09-16T13:01:29.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751823+0000", 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058581190999999998 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.751916+0000", 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3005999999999997e-05 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752030+0000", 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000113204 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54792 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 253952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540320094999998, 2024-09-16T13:01:29.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058958129999999997, 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54792 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693246+0000", 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1899999999999998e-07 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693248+0000", 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693254+0000", 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0129999999999999e-06 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693258+0000", 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5920000000000001e-06 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752078+0000", 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058820482 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752137+0000", 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8387000000000002e-05 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752204+0000", 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7636000000000001e-05 2024-09-16T13:01:29.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54793 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 286720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540304790999997, 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059128760000000002, 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54793 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693261+0000", 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693262+0000", 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693264+0000", 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.809e-06 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693269+0000", 2024-09-16T13:01:29.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4179999999999996e-06 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693273+0000", 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8290000000000001e-06 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752272+0000", 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058999614999999998 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752317+0000", 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4066999999999997e-05 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752390+0000", 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3647999999999997e-05 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54794 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 319488~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.321 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540289353000006, 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059272054999999997, 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54794 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693277+0000", 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693279+0000", 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9120000000000001e-06 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693285+0000", 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8549999999999998e-06 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693289+0000", 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.771e-06 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752430+0000", 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141416000000002 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752482+0000", 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2213000000000003e-05 2024-09-16T13:01:29.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752549+0000", 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6495000000000006e-05 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:54795 4.b 4:d57097cf:::rbd_data.11b58f0917ce.0000000000000002:1a [read 352256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.540273639999995, 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059412815000000001, 2024-09-16T13:01:29.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54795 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.693293+0000", 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707197+0000", 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013904498 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.707204+0000", 2024-09-16T13:01:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711469+0000", 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042645799999999996 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.711483+0000", 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4268e-05 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752589+0000", 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041106281000000001 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752636+0000", 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7351000000000001e-05 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:13.752705+0000", 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8957999999999999e-05 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.330 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.331 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.331 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:29.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:29.333 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:29.323+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:29.433 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:29.427+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:29.436 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:29.436 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:29.436 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:29.445 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:29.534 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:29.527+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:29.593 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:29.593 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:29.594 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:29.594 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:29.594 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:29.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:29.635 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:29.627+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:29.704 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254136 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 136192~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.850047367999998, 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082717923999999998, 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254136 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849491+0000", 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849492+0000", 2024-09-16T13:01:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849494+0000", 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0540000000000002e-06 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849500+0000", 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2899999999999999e-06 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849504+0000", 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8120000000000001e-06 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.931987+0000", 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082483231000000004 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932061+0000", 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4181e-05 2024-09-16T13:01:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932209+0000", 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014796399999999999 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254137 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 139264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.850031092999998, 2024-09-16T13:01:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082930443000000006, 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254137 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849508+0000", 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849510+0000", 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0530000000000001e-06 2024-09-16T13:01:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849516+0000", 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8359999999999997e-06 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849520+0000", 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932239+0000", 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082719191999999997 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932307+0000", 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7824000000000007e-05 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932438+0000", 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000131354 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254138 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 146432~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.850015163000002, 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083100513000000001, 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254138 2024-09-16T13:01:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849524+0000", 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7300000000000001e-07 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849526+0000", 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9089999999999998e-06 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849532+0000", 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6420000000000004e-06 2024-09-16T13:01:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849535+0000", 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7740000000000002e-06 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932467+0000", 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082931469999999993 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932535+0000", 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7815e-05 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932624+0000", 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9430000000000003e-05 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254139 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 149504~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.849999605999997, 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083267068, 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254139 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849539+0000", 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849540+0000", 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849542+0000", 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9379999999999999e-06 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849547+0000", 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6350000000000003e-06 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849551+0000", 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5880000000000002e-06 2024-09-16T13:01:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932652+0000", 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083101176999999998 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932719+0000", 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.703e-05 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932806+0000", 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7340000000000001e-05 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254140 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 156672~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.849984333000002, 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083436222000000004, 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254140 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849555+0000", 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849557+0000", 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8419999999999999e-06 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849564+0000", 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7079999999999997e-06 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849569+0000", 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.386e-06 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932833+0000", 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083264099999999994 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932900+0000", 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7007000000000004e-05 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.932991+0000", 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0815000000000004e-05 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254141 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.849965781000002, 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083599532000000004, 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254141 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849573+0000", 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849574+0000", 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1699999999999998e-07 2024-09-16T13:01:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849576+0000", 2024-09-16T13:01:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-16T13:01:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849582+0000", 2024-09-16T13:01:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9769999999999997e-06 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849585+0000", 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933018+0000", 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083432372000000005 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933086+0000", 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7862999999999997e-05 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933173+0000", 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7050000000000002e-05 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254142 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.849949631000001, 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083767603999999996, 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254142 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849589+0000", 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849590+0000", 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.51e-07 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849592+0000", 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0360000000000001e-06 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849598+0000", 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4139999999999999e-06 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849602+0000", 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.659e-06 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933199+0000", 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083597150999999995 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933267+0000", 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7933000000000002e-05 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933357+0000", 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9859999999999997e-05 2024-09-16T13:01:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254143 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.849932682999999, 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083932426000000004, 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254143 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849606+0000", 2024-09-16T13:01:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849607+0000", 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849609+0000", 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.165e-06 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849615+0000", 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2029999999999997e-06 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849619+0000", 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933382+0000", 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083763447000000005 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933447+0000", 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4993000000000006e-05 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933539+0000", 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1434000000000006e-05 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254145 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.849916121, 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084130242999999993, 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254145 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.849623+0000", 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851596+0000", 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019732550000000001 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851602+0000", 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5720000000000002e-06 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851610+0000", 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5529999999999993e-06 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851616+0000", 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6409999999999999e-06 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933580+0000", 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.081963854000000003 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933653+0000", 2024-09-16T13:01:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3901999999999998e-05 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.933753+0000", 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9465999999999999e-05 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254149 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847899597999998, 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082661199000000005, 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254149 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851639+0000", 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851640+0000", 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851642+0000", 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851648+0000", 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9200000000000001e-06 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851652+0000", 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0439999999999997e-06 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934105+0000", 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082453495000000002 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934193+0000", 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7433000000000001e-05 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934300+0000", 2024-09-16T13:01:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107513 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254151 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847877977000003, 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082910496, 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254151 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851661+0000", 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1799999999999995e-07 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851664+0000", 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4399999999999999e-06 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851670+0000", 2024-09-16T13:01:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3509999999999998e-06 2024-09-16T13:01:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851674+0000", 2024-09-16T13:01:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.912e-06 2024-09-16T13:01:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934385+0000", 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082710396000000005 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934471+0000", 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6040999999999998e-05 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934571+0000", 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100738 2024-09-16T13:01:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254153 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847860523999998, 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083065172000000007, 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254153 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851678+0000", 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851679+0000", 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851681+0000", 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.137e-06 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851687+0000", 2024-09-16T13:01:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.643e-06 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851690+0000", 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5580000000000001e-06 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934597+0000", 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.082906881000000002 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934663+0000", 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5828999999999996e-05 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934743+0000", 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0714e-05 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254155 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847844836, 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083254350000000005, 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254155 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851694+0000", 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-16T13:01:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851697+0000", 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.187e-06 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851702+0000", 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7520000000000002e-06 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851706+0000", 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7979999999999999e-06 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934769+0000", 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083062834000000002 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934829+0000", 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0430999999999998e-05 2024-09-16T13:01:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934948+0000", 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118905 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254157 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 207872~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847828911000001, 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083456394000000003, 2024-09-16T13:01:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254157 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851710+0000", 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851712+0000", 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0150000000000002e-06 2024-09-16T13:01:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851718+0000", 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5999999999999997e-06 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851722+0000", 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5810000000000001e-06 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.934992+0000", 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083270799000000006 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935066+0000", 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3851999999999997e-05 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935166+0000", 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100145 2024-09-16T13:01:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254159 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 210944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847813565000003, 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083625606000000005, 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254159 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851725+0000", 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851726+0000", 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851728+0000", 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851734+0000", 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7429999999999999e-06 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851738+0000", 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9620000000000004e-06 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935195+0000", 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083457518999999994 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935262+0000", 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6465999999999997e-05 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935351+0000", 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9314000000000006e-05 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254161 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 218112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847797575000001, 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083866861000000001, 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254161 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851741+0000", 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851742+0000", 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-16T13:01:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851744+0000", 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1550000000000001e-06 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851749+0000", 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5829999999999997e-06 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851753+0000", 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6990000000000001e-06 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935377+0000", 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083624025000000005 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935454+0000", 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7168000000000004e-05 2024-09-16T13:01:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935608+0000", 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153774 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254163 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 221184~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847781799000003, 2024-09-16T13:01:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084055344000000004, 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254163 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851757+0000", 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.89e-07 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851760+0000", 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2249999999999999e-06 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851765+0000", 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7799999999999997e-06 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851769+0000", 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.72e-06 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935654+0000", 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.083884829999999994 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935723+0000", 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9163000000000002e-05 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.935812+0000", 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9237e-05 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254164 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847765756000001, 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084726810999999999, 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254164 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851773+0000", 2024-09-16T13:01:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851775+0000", 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.945e-06 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851781+0000", 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851785+0000", 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6320000000000001e-06 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936053+0000", 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084268018 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936130+0000", 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7478999999999995e-05 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936500+0000", 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036965099999999997 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254165 4.f 4:f02c72b2:::rbd_data.11b55b44334a.0000000000000020:23 [read 228352~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847750232000003, 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084960919999999995, 2024-09-16T13:01:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254165 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851789+0000", 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851791+0000", 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2819999999999999e-06 2024-09-16T13:01:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851797+0000", 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7969999999999997e-06 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851801+0000", 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6679999999999999e-06 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936554+0000", 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.084753203999999999 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936625+0000", 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1406e-05 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936750+0000", 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012416700000000001 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:254166 4.f 4:f4398d9a:::rbd_data.11b55b44334a.0000000000000001:23 [read 6144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.847734420999998, 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.085115273000000005, 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254166 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.851804+0000", 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856445+0000", 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046400929999999996 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856451+0000", 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5119999999999997e-06 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856463+0000", 2024-09-16T13:01:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2252999999999999e-05 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.856470+0000", 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4160000000000001e-06 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936767+0000", 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080297258999999996 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936832+0000", 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4685999999999997e-05 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:43.936920+0000", 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8053999999999994e-05 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:29.798 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:29.798 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:29.799 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:29.799 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:29.796+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:29.900 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:29.891+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:29.949 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T13:01:29.966 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:29.967 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:29.967 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:29.976 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:30.021 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:30.015+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:30.075 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:30.075 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:30.075 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:30.085 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:30.102 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:30.096+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:30.185 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:30.185 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:30.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:30.185 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260177 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 159744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105857124000003, 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039279541000000001, 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260177 2024-09-16T13:01:30.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075844+0000", 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075845+0000", 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075847+0000", 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075852+0000", 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.507e-06 2024-09-16T13:01:30.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075856+0000", 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6389999999999998e-06 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.114965+0000", 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039109496000000001 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115034+0000", 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8378000000000002e-05 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115124+0000", 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0334999999999995e-05 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.190 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260178 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 166912~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105841728000001, 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039530044, 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260178 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075860+0000", 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075866+0000", 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0909999999999998e-06 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075872+0000", 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.139e-06 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075876+0000", 2024-09-16T13:01:30.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0419999999999996e-06 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115194+0000", 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039317193 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115259+0000", 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5399000000000002e-05 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115390+0000", 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130803 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260179 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 169984~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105820878000003, 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039926426000000001, 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260179 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075881+0000", 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-07 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075883+0000", 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6959999999999999e-06 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075889+0000", 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9479999999999996e-06 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075893+0000", 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7840000000000001e-06 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115419+0000", 2024-09-16T13:01:30.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039526517999999997 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115577+0000", 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015783999999999999 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115807+0000", 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023010399999999999 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260180 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 177152~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105805130999997, 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040245398000000002, 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260180 2024-09-16T13:01:30.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075896+0000", 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075897+0000", 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1800000000000001e-07 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075899+0000", 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8169999999999999e-06 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075904+0000", 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075908+0000", 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5889999999999999e-06 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115837+0000", 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.039929193000000002 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.115978+0000", 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014115800000000001 2024-09-16T13:01:30.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116142+0000", 2024-09-16T13:01:30.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016352900000000001 2024-09-16T13:01:30.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.204 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.204 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260181 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 180224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105789924, 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040482388000000001, 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260181 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075912+0000", 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-16T13:01:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075914+0000", 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6759999999999999e-06 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075919+0000", 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2360000000000004e-06 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075922+0000", 2024-09-16T13:01:30.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116171+0000", 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040248702999999997 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116239+0000", 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8391000000000003e-05 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116394+0000", 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000154575 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260182 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 187392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105775213000001, 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042150168000000002, 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260182 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075926+0000", 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075927+0000", 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075929+0000", 2024-09-16T13:01:30.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7859999999999999e-06 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075934+0000", 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7679999999999997e-06 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075938+0000", 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2590000000000001e-06 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116426+0000", 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040488077999999997 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116493+0000", 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7681000000000005e-05 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118077+0000", 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015832260000000001 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260183 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 190464~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105760224999997, 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040886082999999997, 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260183 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075941+0000", 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075942+0000", 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075943+0000", 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.778e-06 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075949+0000", 2024-09-16T13:01:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075953+0000", 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116672+0000", 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040719563 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116743+0000", 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0729000000000007e-05 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116827+0000", 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4364999999999996e-05 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260184 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 197632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.218 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105745061999997, 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041043524999999997, 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260184 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075957+0000", 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-16T13:01:30.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075959+0000", 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6869999999999999e-06 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075964+0000", 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.242e-06 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075967+0000", 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116850+0000", 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.040882977000000001 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.116916+0000", 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5413000000000005e-05 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117000+0000", 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4431999999999994e-05 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260185 4.1c 4:3cd52135:::rbd_data.11b55b44334a.0000000000000045:23 [read 200704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:30.223 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105729822999997, 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041177895999999999, 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260185 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075972+0000", 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-16T13:01:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075974+0000", 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0049999999999999e-06 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075980+0000", 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7470000000000002e-06 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.075984+0000", 2024-09-16T13:01:30.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6899999999999998e-06 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117022+0000", 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041038789999999999 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117090+0000", 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7712000000000005e-05 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117150+0000", 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9542e-05 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260257 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105687969000002, 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041222406000000003, 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260257 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:30.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076014+0000", 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076017+0000", 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4990000000000001e-06 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076024+0000", 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076028+0000", 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9319999999999998e-06 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117162+0000", 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04113472 2024-09-16T13:01:30.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117199+0000", 2024-09-16T13:01:30.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6507999999999997e-05 2024-09-16T13:01:30.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117236+0000", 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7023000000000001e-05 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260258 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105669450000001, 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041343291999999997, 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.233 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260258 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076032+0000", 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076035+0000", 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.058e-06 2024-09-16T13:01:30.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076041+0000", 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.998e-06 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076044+0000", 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3900000000000002e-06 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117256+0000", 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041212555999999997 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117323+0000", 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6568000000000004e-05 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117375+0000", 2024-09-16T13:01:30.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2361999999999998e-05 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260259 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105654031999997, 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041437587999999997, 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260259 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076048+0000", 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5499999999999999e-07 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076050+0000", 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9259999999999999e-06 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076055+0000", 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5609999999999998e-06 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076059+0000", 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3739999999999998e-06 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117396+0000", 2024-09-16T13:01:30.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041336879999999999 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117454+0000", 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8443999999999998e-05 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117485+0000", 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1047999999999998e-05 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260260 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105639232999998, 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041496984000000001, 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260260 2024-09-16T13:01:30.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076062+0000", 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076063+0000", 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5900000000000003e-07 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076070+0000", 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7919999999999993e-06 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076076+0000", 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9359999999999996e-06 2024-09-16T13:01:30.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076080+0000", 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8240000000000001e-06 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117497+0000", 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041416896000000002 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117531+0000", 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4e-05 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117559+0000", 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8177000000000001e-05 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260261 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.247 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105616996999998, 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041555793000000001, 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260261 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076085+0000", 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076087+0000", 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7850000000000001e-06 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076093+0000", 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6879999999999996e-06 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076096+0000", 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3210000000000001e-06 2024-09-16T13:01:30.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117570+0000", 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041474329999999997 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117611+0000", 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0852999999999997e-05 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117640+0000", 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9213e-05 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260262 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105601550999999, 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041643056999999997, 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260262 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076100+0000", 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076102+0000", 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8509999999999999e-06 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076108+0000", 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8560000000000003e-06 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076111+0000", 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0620000000000001e-06 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117651+0000", 2024-09-16T13:01:30.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041539980999999997 2024-09-16T13:01:30.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117703+0000", 2024-09-16T13:01:30.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1975000000000003e-05 2024-09-16T13:01:30.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117743+0000", 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9872e-05 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260263 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.10558657, 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041724412000000002, 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.257 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260263 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076115+0000", 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076117+0000", 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.762e-06 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076123+0000", 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6980000000000003e-06 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076126+0000", 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.427e-06 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117757+0000", 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041631230999999998 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117792+0000", 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4078999999999997e-05 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117839+0000", 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7861000000000001e-05 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.261 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260264 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105571724000001, 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041783638999999997, 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260264 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076130+0000", 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076132+0000", 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.751e-06 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076138+0000", 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6729999999999997e-06 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076141+0000", 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1930000000000002e-06 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117851+0000", 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04171064 2024-09-16T13:01:30.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117885+0000", 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3807999999999999e-05 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117913+0000", 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8229999999999999e-05 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260265 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105557120999997, 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042095026000000001, 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260265 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076144+0000", 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076145+0000", 2024-09-16T13:01:30.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4299999999999999e-07 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076147+0000", 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9520000000000001e-06 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076152+0000", 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5130000000000004e-06 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076156+0000", 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5719999999999999e-06 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117924+0000", 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041768179000000002 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.117980+0000", 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6125000000000002e-05 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.118239+0000", 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025934200000000003 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260266 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105542253000003, 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045273031999999998, 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260266 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076159+0000", 2024-09-16T13:01:30.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076160+0000", 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076161+0000", 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.796e-06 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076167+0000", 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7139999999999998e-06 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076171+0000", 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.579e-06 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.120966+0000", 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.044795297999999997 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121159+0000", 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192618 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.121432+0000", 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000273663 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:260267 4.1c 4:3cbf3b5f:::rbd_data.11b55b44334a.000000000000009a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.105525876000002, 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046181535000000003, 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 260267 2024-09-16T13:01:30.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076176+0000", 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076178+0000", 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7969999999999999e-06 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076184+0000", 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6570000000000002e-06 2024-09-16T13:01:30.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.076187+0000", 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.455e-06 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122253+0000", 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046065813999999997 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122318+0000", 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5338000000000004e-05 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:44.122357+0000", 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9088999999999998e-05 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.280 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:30.282 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:30.282 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:30.275+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:30.382 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:30.375+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:30.432 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:30.432 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:30.442 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:30.483 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:30.475+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:30.540 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:30.540 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:30.549 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:30.584 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:30.580+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:30.685 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:30.679+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:30.700 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.627096999000003, 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:01:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:01:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:01:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:01:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:01:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.627044580000003, 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:01:30.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:01:30.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.627027732000002, 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:30.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:30.714 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.627011617999997, 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:01:30.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:01:30.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.626946688000004, 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:01:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:01:30.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.626930588999997, 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:01:30.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.626856807999999, 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.728 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:01:30.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.626839606999994, 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:01:30.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.626822903999994, 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:01:30.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:01:30.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.090511649, 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:01:30.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:01:30.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.090494513000003, 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.751 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.090474931000003, 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:01:30.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:01:30.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.090444071999997, 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:30.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.090375975999997, 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:01:30.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:01:30.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.083072121000001, 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:01:30.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:01:30.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.083054531999998, 2024-09-16T13:01:30.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:01:30.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.773 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.083039638999999, 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:01:30.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:01:30.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:30.778 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.083008022000001, 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:01:30.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:01:30.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.082990000999999, 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:01:30.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:30.787 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.168954628999998, 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2569454119999999, 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 279128 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527093+0000", 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4040000000000002e-06 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527104+0000", 2024-09-16T13:01:30.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-05 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527133+0000", 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8909999999999999e-05 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527144+0000", 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1497e-05 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527223+0000", 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8577000000000003e-05 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527236+0000", 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3362e-05 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783387+0000", 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2561508619999999 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:30.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783699+0000", 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311687 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.784036+0000", 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033771300000000001 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:30.792 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:30.794 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:30.794 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:30.788+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:30.895 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:30.891+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:30.904 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:30.905 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:30.905 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:30.905 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:30.914 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:30.930 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:01:30.996 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:30.987+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:31.017 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:31.017 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:31.017 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:31.017 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:31.017 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:31.027 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:31.080 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:31.096 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:31.092+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:31.106 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:31.127 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240664 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 152576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.583857571999999, 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047790817999999999, 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.131 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240664 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1900000000000003e-07 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543064+0000", 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4449999999999999e-06 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543079+0000", 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5221e-05 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543084+0000", 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8590000000000004e-06 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590750+0000", 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047665710999999999 2024-09-16T13:01:31.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590801+0000", 2024-09-16T13:01:31.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1827999999999999e-05 2024-09-16T13:01:31.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590852+0000", 2024-09-16T13:01:31.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0235000000000003e-05 2024-09-16T13:01:31.137 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.137 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.137 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.138 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240665 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.138 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:31.138 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.583828169, 2024-09-16T13:01:31.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047896192999999997, 2024-09-16T13:01:31.138 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.138 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.139 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240665 2024-09-16T13:01:31.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.139 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:31.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:31.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543091+0000", 2024-09-16T13:01:31.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:01:31.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543094+0000", 2024-09-16T13:01:31.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7149999999999998e-06 2024-09-16T13:01:31.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543102+0000", 2024-09-16T13:01:31.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6549999999999993e-06 2024-09-16T13:01:31.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543108+0000", 2024-09-16T13:01:31.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4840000000000003e-06 2024-09-16T13:01:31.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590885+0000", 2024-09-16T13:01:31.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047777396 2024-09-16T13:01:31.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590936+0000", 2024-09-16T13:01:31.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0611e-05 2024-09-16T13:01:31.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590987+0000", 2024-09-16T13:01:31.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0785000000000003e-05 2024-09-16T13:01:31.147 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.147 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.148 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.148 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240666 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 162816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.148 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:31.148 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.583805900999998, 2024-09-16T13:01:31.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048010069000000002, 2024-09-16T13:01:31.149 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.149 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.149 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.149 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.150 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240666 2024-09-16T13:01:31.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.150 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:31.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:31.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:31.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8500000000000002e-07 2024-09-16T13:01:31.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543116+0000", 2024-09-16T13:01:31.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.531e-06 2024-09-16T13:01:31.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543123+0000", 2024-09-16T13:01:31.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1069999999999999e-06 2024-09-16T13:01:31.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543127+0000", 2024-09-16T13:01:31.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4309999999999996e-06 2024-09-16T13:01:31.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591020+0000", 2024-09-16T13:01:31.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047892413000000002 2024-09-16T13:01:31.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591071+0000", 2024-09-16T13:01:31.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1650000000000002e-05 2024-09-16T13:01:31.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591123+0000", 2024-09-16T13:01:31.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1452000000000001e-05 2024-09-16T13:01:31.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.158 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.159 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240667 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.159 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:31.159 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.583787119, 2024-09-16T13:01:31.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048125951, 2024-09-16T13:01:31.159 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.160 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.160 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240667 2024-09-16T13:01:31.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.161 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:31.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:31.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543132+0000", 2024-09-16T13:01:31.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5099999999999999e-07 2024-09-16T13:01:31.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543134+0000", 2024-09-16T13:01:31.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1500000000000002e-06 2024-09-16T13:01:31.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543141+0000", 2024-09-16T13:01:31.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:31.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543146+0000", 2024-09-16T13:01:31.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9200000000000003e-06 2024-09-16T13:01:31.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591158+0000", 2024-09-16T13:01:31.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048011246 2024-09-16T13:01:31.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591207+0000", 2024-09-16T13:01:31.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9295999999999997e-05 2024-09-16T13:01:31.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591257+0000", 2024-09-16T13:01:31.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0426000000000001e-05 2024-09-16T13:01:31.168 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.169 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.169 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.169 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240668 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 173056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.169 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:31.170 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.583767414, 2024-09-16T13:01:31.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048243280999999999, 2024-09-16T13:01:31.170 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.170 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.170 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.171 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.171 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.171 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240668 2024-09-16T13:01:31.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.171 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:31.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:31.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543152+0000", 2024-09-16T13:01:31.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:01:31.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543154+0000", 2024-09-16T13:01:31.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5380000000000001e-06 2024-09-16T13:01:31.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543161+0000", 2024-09-16T13:01:31.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7920000000000004e-06 2024-09-16T13:01:31.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543172+0000", 2024-09-16T13:01:31.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1327e-05 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591290+0000", 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048118181000000003 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591344+0000", 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4061000000000003e-05 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591394+0000", 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9866e-05 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240766 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.582563088999997, 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047716644000000003, 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.177 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240766 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:31.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544356+0000", 2024-09-16T13:01:31.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2500000000000005e-07 2024-09-16T13:01:31.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544373+0000", 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6685999999999999e-05 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544381+0000", 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7190000000000001e-06 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544384+0000", 2024-09-16T13:01:31.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591969+0000", 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047585483999999997 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592021+0000", 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1839000000000003e-05 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592072+0000", 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754000000000002e-05 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240767 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 201728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.582531213999999, 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047822479000000001, 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240767 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544388+0000", 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.51e-07 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544390+0000", 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.824e-06 2024-09-16T13:01:31.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544393+0000", 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.681e-06 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544396+0000", 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4940000000000002e-06 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592105+0000", 2024-09-16T13:01:31.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047709184000000002 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592158+0000", 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2772999999999998e-05 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592210+0000", 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2071999999999999e-05 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240768 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.582518413000003, 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047944647, 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240768 2024-09-16T13:01:31.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0400000000000002e-07 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544403+0000", 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1430000000000001e-06 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544406+0000", 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7450000000000001e-06 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544409+0000", 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592241+0000", 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047832146999999998 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592291+0000", 2024-09-16T13:01:31.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9289000000000002e-05 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592345+0000", 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4067000000000003e-05 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240769 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.582506788000003, 2024-09-16T13:01:31.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048066225999999997, 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240769 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1399999999999998e-07 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544413+0000", 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.42e-06 2024-09-16T13:01:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544417+0000", 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.529e-06 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544420+0000", 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5620000000000002e-06 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592378+0000", 2024-09-16T13:01:31.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047958354000000002 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592427+0000", 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9013e-05 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592478+0000", 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1034e-05 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240770 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.582496525000003, 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048189432999999997, 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240770 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3200000000000001e-07 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544424+0000", 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4449999999999999e-06 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544427+0000", 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5049999999999999e-06 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544430+0000", 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.61e-06 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592511+0000", 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048080912000000003 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592563+0000", 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2107000000000001e-05 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592611+0000", 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8522000000000001e-05 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240771 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.582461918, 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290128000000002, 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.200 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240771 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5700000000000002e-07 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544459+0000", 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6840000000000001e-06 2024-09-16T13:01:31.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544466+0000", 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999997e-06 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544469+0000", 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592645+0000", 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048175216999999999 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592696+0000", 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1292999999999998e-05 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592747+0000", 2024-09-16T13:01:31.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0751000000000002e-05 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240772 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.582438623000002, 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048403333999999999, 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240772 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544482+0000", 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8390000000000001e-06 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544486+0000", 2024-09-16T13:01:31.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9369999999999997e-06 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544489+0000", 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8459999999999999e-06 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592779+0000", 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290301000000001 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592831+0000", 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2335e-05 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592883+0000", 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1786999999999997e-05 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240773 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.209 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.582427238000001, 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048527963, 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240773 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:31.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544492+0000", 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.84e-07 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544493+0000", 2024-09-16T13:01:31.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6279999999999999e-06 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544498+0000", 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4689999999999999e-06 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544500+0000", 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592916+0000", 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048415672 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592965+0000", 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9120999999999999e-05 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593019+0000", 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3950999999999999e-05 2024-09-16T13:01:31.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240774 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.582415583, 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048649546000000002, 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240774 2024-09-16T13:01:31.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544505+0000", 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.61e-06 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544508+0000", 2024-09-16T13:01:31.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4929999999999998e-06 2024-09-16T13:01:31.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544511+0000", 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4169999999999999e-06 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593051+0000", 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048539604 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593103+0000", 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2068999999999999e-05 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593152+0000", 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9870999999999997e-05 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240775 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.582405328999997, 2024-09-16T13:01:31.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048773328999999997, 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240775 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545618+0000", 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011045810000000001 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545625+0000", 2024-09-16T13:01:31.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6079999999999998e-06 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545644+0000", 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.823e-05 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545653+0000", 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3000000000000007e-06 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593186+0000", 2024-09-16T13:01:31.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047533387000000003 2024-09-16T13:01:31.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593235+0000", 2024-09-16T13:01:31.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8767000000000003e-05 2024-09-16T13:01:31.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593286+0000", 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1456000000000003e-05 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240776 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.581253191000002, 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047754889000000002, 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.226 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240776 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545666+0000", 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-06 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545671+0000", 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2270000000000004e-06 2024-09-16T13:01:31.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545681+0000", 2024-09-16T13:01:31.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0506000000000001e-05 2024-09-16T13:01:31.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545687+0000", 2024-09-16T13:01:31.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0970000000000002e-06 2024-09-16T13:01:31.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593320+0000", 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047632999000000002 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593369+0000", 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8882999999999999e-05 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593420+0000", 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1127e-05 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240777 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.581224570000003, 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047878318000000003, 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.231 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240777 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545695+0000", 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545698+0000", 2024-09-16T13:01:31.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0709999999999999e-06 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545707+0000", 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7629999999999995e-06 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545713+0000", 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2839999999999997e-06 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593453+0000", 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047740281000000002 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593518+0000", 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5493000000000005e-05 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593572+0000", 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3739000000000002e-05 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.236 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240778 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.236 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:31.236 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.581200131000003, 2024-09-16T13:01:31.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047999411999999998, 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240778 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545719+0000", 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.13e-07 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545728+0000", 2024-09-16T13:01:31.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1560000000000001e-06 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545901+0000", 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017311599999999999 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545909+0000", 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.028e-06 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593606+0000", 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047696660000000002 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593666+0000", 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9611999999999998e-05 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593718+0000", 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2227e-05 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.240 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240951 4.8 4:10cc0d5f:::rbd_data.11b552946c0f.0000000000000120:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.573119861999999, 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048226894999999999, 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240951 2024-09-16T13:01:31.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2300000000000002e-07 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553801+0000", 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.686e-06 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553805+0000", 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3039999999999998e-06 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553809+0000", 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.599994+0000", 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046185154999999999 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.600112+0000", 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118589 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.602026+0000", 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019131370000000001 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:272626 4.12 4:4935e950:::rbd_data.11b59088eb24.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.669092239999998, 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022439990000001, 2024-09-16T13:01:31.245 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 272626 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457828+0000", 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457835+0000", 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7900000000000002e-06 2024-09-16T13:01:31.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457853+0000", 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7671000000000001e-05 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457859+0000", 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2949999999999999e-06 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457913+0000", 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4112e-05 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457922+0000", 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0850000000000003e-06 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.457879+0000", 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.99995641700000004 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458079+0000", 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020072300000000001 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458304+0000", 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000224601 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459393+0000", 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010886170000000001 2024-09-16T13:01:31.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459877+0000", 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048443500000000002 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460006+0000", 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012914 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460070+0000", 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4074999999999999e-05 2024-09-16T13:01:31.251 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.252 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:31.252 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:31.252 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:31.253 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:31.254 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:31.254 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:31.254 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:31.254 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:31.255 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:31.255 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:31.248+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:31.256 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:31.256 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:31.256 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:31.256 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:31.256 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:31.256 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:31.256 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:31.256 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:31.256 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:31.355 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:31.347+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:31.406 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:31.407 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:31.417 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:31.455 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:31.447+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:31.515 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:31.515 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:31.525 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:31.556 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:31.551+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:31.620 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:31.621 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:31.630 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:31.657 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:31.651+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:31.734 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:31.734 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:31.743 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:31.758 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:31.751+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:31.843 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:31.843 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:31.852 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:31.859 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:31.855+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:31.951 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:31.951 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:31.960 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:31.951+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:31.961 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:31.965 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T13:01:32.061 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:32.056+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:32.069 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:32.070 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:32.081 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:32.162 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:32.155+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:32.181 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:32.181 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:32.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:32.263 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:32.260+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:32.340 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:32.340 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:32.349 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:32.364 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:32.360+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:32.448 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:32.449 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:32.457 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:32.464 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:32.455+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:32.548 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:32.549 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:32.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:32.565 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:32.560+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:32.660 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:32.656+0000 7f90445c1540 -1 osd.6 120 log_to_monitors true 2024-09-16T13:01:32.666 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:32.659+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:32.704 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:32.704 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:32.713 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:32.767 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:32.764+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:32.814 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:32.814 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:32.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:32.868 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:32.864+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:32.923 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:32.923 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:32.932 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:32.969 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:32.964+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:32.975 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T13:01:33.000 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:32.996+0000 7f9038b04700 -1 osd.6 146 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T13:01:33.053 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:33.053 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:33.063 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:33.069 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:33.063+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:33.166 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:33.167 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:33.170 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:33.164+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:33.176 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:33.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:01:33.270 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:33.263+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:33.308 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:01:33.320 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:33.320 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:33.329 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:33.371 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:33.368+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:33.391 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:33.419 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:33.420 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:33.429 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T13:01:33.430 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:33.466 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:33.472 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:33.463+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:33.488 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:33.503 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-16T13:01:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-16T13:01:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-16T13:01:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97466 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4978688~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:33.503 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 68.429289850999993, 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070470891999999993, 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97466 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068949+0000", 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9999999999999997e-07 2024-09-16T13:01:33.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068951+0000", 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0669999999999999e-06 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068957+0000", 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0240000000000003e-06 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.068961+0000", 2024-09-16T13:01:33.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0450000000000002e-06 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139237+0000", 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070275264000000004 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139311+0000", 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4740000000000006e-05 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139420+0000", 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108152 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97486 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4757504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 68.429237431999994, 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070641474999999995, 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97486 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069001+0000", 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069003+0000", 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9250000000000002e-06 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069010+0000", 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069014+0000", 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6370000000000001e-06 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139454+0000", 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070440738000000003 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139533+0000", 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8685999999999998e-05 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139643+0000", 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010937099999999999 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.512 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97487 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4847616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 68.429220584000007, 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070864786999999999, 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97487 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069018+0000", 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069020+0000", 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9489999999999999e-06 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069026+0000", 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6239999999999999e-06 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069030+0000", 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0670000000000002e-06 2024-09-16T13:01:33.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139669+0000", 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070638909 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139746+0000", 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6762999999999997e-05 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139883+0000", 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137133 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97488 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4937728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 68.429204470000002, 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071070526999999994, 2024-09-16T13:01:33.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97488 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069034+0000", 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5199999999999998e-07 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069036+0000", 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9850000000000001e-06 2024-09-16T13:01:33.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069043+0000", 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5010000000000001e-06 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069046+0000", 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5700000000000001e-06 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139908+0000", 2024-09-16T13:01:33.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070861782999999998 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.139985+0000", 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7157999999999995e-05 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140105+0000", 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119178 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97511 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4806656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 68.429139539999994, 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210384000000002, 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97511 2024-09-16T13:01:33.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069099+0000", 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5400000000000002e-07 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069101+0000", 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0080000000000001e-06 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069107+0000", 2024-09-16T13:01:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6289999999999998e-06 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069111+0000", 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9770000000000002e-06 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140130+0000", 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071019013000000006 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140203+0000", 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2719000000000006e-05 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140309+0000", 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106684 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97512 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4896768~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 68.429123441000002, 2024-09-16T13:01:33.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071406810000000001, 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97512 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069115+0000", 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069116+0000", 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0999999999999998e-07 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069118+0000", 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4509999999999999e-06 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069124+0000", 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4689999999999996e-06 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069127+0000", 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140337+0000", 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071210184999999995 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140412+0000", 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4462000000000006e-05 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140522+0000", 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110099 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97532 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4765696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 68.429049660000004, 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071542381000000002, 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97532 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069189+0000", 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069191+0000", 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0449999999999999e-06 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069198+0000", 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7859999999999999e-06 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069202+0000", 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.709e-06 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140548+0000", 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071345934 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140624+0000", 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6475000000000002e-05 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140731+0000", 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107067 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97533 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4855808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 68.429032458999998, 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071764009000000004, 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97533 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069206+0000", 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069208+0000", 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069214+0000", 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0039999999999996e-06 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069218+0000", 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9179999999999996e-06 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140755+0000", 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071536630000000004 2024-09-16T13:01:33.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140826+0000", 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609000000000001e-05 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140970+0000", 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014447900000000001 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:97534 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4945920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 68.429015755999998, 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071956346000000004, 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 97534 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069223+0000", 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6700000000000003e-07 2024-09-16T13:01:33.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069225+0000", 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069233+0000", 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5320000000000001e-06 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.069238+0000", 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0889999999999999e-06 2024-09-16T13:01:33.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.140996+0000", 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071758341000000003 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141070+0000", 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3831000000000005e-05 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:25.141179+0000", 2024-09-16T13:01:33.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108916 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159302 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2895872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.892704500999997, 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071290929000000003, 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159302 2024-09-16T13:01:33.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.545 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:33.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605534+0000", 2024-09-16T13:01:33.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605535+0000", 2024-09-16T13:01:33.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-16T13:01:33.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605537+0000", 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310000000000001e-06 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605543+0000", 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605547+0000", 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6349999999999999e-06 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672610+0000", 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.067063333000000003 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.672686+0000", 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6254000000000005e-05 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676825+0000", 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041384389999999998 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159304 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2940928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.892687365, 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071522366000000004, 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159304 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605551+0000", 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605552+0000", 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605557+0000", 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9180000000000002e-06 2024-09-16T13:01:33.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605563+0000", 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8789999999999997e-06 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605567+0000", 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.631e-06 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676862+0000", 2024-09-16T13:01:33.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071295226000000003 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.676948+0000", 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6227e-05 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677074+0000", 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125031 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159306 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 2985984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.892667782999993, 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071727558999999996, 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159306 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605571+0000", 2024-09-16T13:01:33.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605573+0000", 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8190000000000001e-06 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605579+0000", 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6160000000000001e-06 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605582+0000", 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.461e-06 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677099+0000", 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071516827000000005 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677179+0000", 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0161999999999996e-05 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677298+0000", 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119296 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159308 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3031040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.892636924000001, 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071916382000000001, 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159308 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605602+0000", 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605650+0000", 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8239999999999999e-05 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605653+0000", 2024-09-16T13:01:33.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9730000000000002e-06 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605660+0000", 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6399999999999997e-06 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605665+0000", 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677324+0000", 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071658856000000007 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677402+0000", 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8611999999999999e-05 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677518+0000", 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115606 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159309 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.892568827999995, 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.072065590999999998, 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159309 2024-09-16T13:01:33.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.605670+0000", 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609933+0000", 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004263111 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.609938+0000", 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3179999999999997e-06 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611693+0000", 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001754743 2024-09-16T13:01:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.611718+0000", 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5616e-05 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677543+0000", 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.065824611000000005 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677620+0000", 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6995000000000004e-05 2024-09-16T13:01:33.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.677735+0000", 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115197 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159503 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 8101888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:33.568 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.885264973000005, 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070749061000000002, 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159503 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612974+0000", 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612976+0000", 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-16T13:01:33.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612982+0000", 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.801e-06 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612987+0000", 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3120000000000001e-06 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683345+0000", 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070357852999999998 2024-09-16T13:01:33.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683472+0000", 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000127362 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683723+0000", 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025055500000000002 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159504 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4538368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.885247383999996, 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070971083000000004, 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159504 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612991+0000", 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3799999999999998e-07 2024-09-16T13:01:33.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612993+0000", 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8789999999999999e-06 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.612999+0000", 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3619999999999997e-06 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613002+0000", 2024-09-16T13:01:33.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683753+0000", 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070750857 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683839+0000", 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6396000000000004e-05 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683962+0000", 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012281299999999999 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159506 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4583424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.885232490999996, 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071221924000000006, 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159506 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613006+0000", 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4999999999999998e-07 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613008+0000", 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.86e-06 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613014+0000", 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5280000000000003e-06 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613017+0000", 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3119999999999998e-06 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.683993+0000", 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.070975616000000005 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684077+0000", 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4357000000000006e-05 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684228+0000", 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150901 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159508 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4628480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:33.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.885200874000006, 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071431906000000003, 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159508 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613038+0000", 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613041+0000", 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4770000000000002e-06 2024-09-16T13:01:33.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613047+0000", 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.691e-06 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613051+0000", 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684258+0000", 2024-09-16T13:01:33.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.07120638 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684341+0000", 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2917000000000006e-05 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684470+0000", 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012901199999999999 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:159509 4.1b 4:db8a7363:::rbd_data.11b5c2ee3542.0000000000000002:1d [read 4673536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.885182853000003, 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071689621999999995, 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 159509 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:33.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613056+0000", 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613058+0000", 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9099999999999999e-06 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613064+0000", 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8989999999999998e-06 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.613067+0000", 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4209999999999999e-06 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684498+0000", 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071430545999999998 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684621+0000", 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012306900000000001 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:28.684745+0000", 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012439800000000001 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:33.590 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.971147481000003, 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2569454119999999, 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4533", 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 279128 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527091+0000", 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527093+0000", 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4040000000000002e-06 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527104+0000", 2024-09-16T13:01:33.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.04e-05 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527133+0000", 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8909999999999999e-05 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527144+0000", 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1497e-05 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527223+0000", 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8577000000000003e-05 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:56.527236+0000", 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3362e-05 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783387+0000", 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2561508619999999 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-16T13:01:33.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.783699+0000", 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311687 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-16T13:00:57.784036+0000", 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033771300000000001 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:33.595 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:33.596 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:33.597 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:33.597 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:33.598 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T13:01:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:33.598 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T13:01:33.599 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:33.599 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T13:01:33.599 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:33.587+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:33.698 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:33.692+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:33.708 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:33.709 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:33.718 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:33.798 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:33.796+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:33.863 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:33.864 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:33.864 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:33.864 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:33.864 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:33.873 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:33.919 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:33.916+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:33.983 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T13:01:33.983 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240664 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 152576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.436615432000004, 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047790817999999999, 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240664 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543061+0000", 2024-09-16T13:01:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1900000000000003e-07 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543064+0000", 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4449999999999999e-06 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543079+0000", 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5221e-05 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543084+0000", 2024-09-16T13:01:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8590000000000004e-06 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590750+0000", 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047665710999999999 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590801+0000", 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1827999999999999e-05 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590852+0000", 2024-09-16T13:01:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0235000000000003e-05 2024-09-16T13:01:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240665 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.436586028999997, 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047896192999999997, 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240665 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543090+0000", 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543091+0000", 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4700000000000001e-07 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543094+0000", 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7149999999999998e-06 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543102+0000", 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6549999999999993e-06 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543108+0000", 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4840000000000003e-06 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590885+0000", 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047777396 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590936+0000", 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0611e-05 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.590987+0000", 2024-09-16T13:01:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0785000000000003e-05 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240666 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 162816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.436563761000002, 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048010069000000002, 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240666 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543113+0000", 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8500000000000002e-07 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543116+0000", 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.531e-06 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543123+0000", 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1069999999999999e-06 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543127+0000", 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4309999999999996e-06 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591020+0000", 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047892413000000002 2024-09-16T13:01:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591071+0000", 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1650000000000002e-05 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591123+0000", 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1452000000000001e-05 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240667 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.436544978999997, 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048125951, 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240667 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543131+0000", 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543132+0000", 2024-09-16T13:01:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5099999999999999e-07 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543134+0000", 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1500000000000002e-06 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543141+0000", 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2620000000000001e-06 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543146+0000", 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9200000000000003e-06 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591158+0000", 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048011246 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591207+0000", 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9295999999999997e-05 2024-09-16T13:01:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591257+0000", 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0426000000000001e-05 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240668 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 173056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.436525273999997, 2024-09-16T13:01:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048243280999999999, 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240668 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543151+0000", 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543152+0000", 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1600000000000001e-07 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543154+0000", 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5380000000000001e-06 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543161+0000", 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7920000000000004e-06 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.543172+0000", 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1327e-05 2024-09-16T13:01:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591290+0000", 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048118181000000003 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591344+0000", 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4061000000000003e-05 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591394+0000", 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9866e-05 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240766 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.435320949000001, 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047716644000000003, 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240766 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544355+0000", 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544356+0000", 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2500000000000005e-07 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544373+0000", 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6685999999999999e-05 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544381+0000", 2024-09-16T13:01:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7190000000000001e-06 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544384+0000", 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4369999999999999e-06 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.591969+0000", 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047585483999999997 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592021+0000", 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1839000000000003e-05 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592072+0000", 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754000000000002e-05 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240767 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 201728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.435289074000003, 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047822479000000001, 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240767 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544387+0000", 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544388+0000", 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.51e-07 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544390+0000", 2024-09-16T13:01:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.824e-06 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544393+0000", 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.681e-06 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544396+0000", 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4940000000000002e-06 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592105+0000", 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047709184000000002 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592158+0000", 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2772999999999998e-05 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592210+0000", 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2071999999999999e-05 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240768 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.435276272999999, 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047944647, 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240768 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544400+0000", 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0400000000000002e-07 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544403+0000", 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1430000000000001e-06 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544406+0000", 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7450000000000001e-06 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544409+0000", 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592241+0000", 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047832146999999998 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592291+0000", 2024-09-16T13:01:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9289000000000002e-05 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592345+0000", 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4067000000000003e-05 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240769 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 211968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.435264648, 2024-09-16T13:01:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048066225999999997, 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240769 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544412+0000", 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1399999999999998e-07 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544413+0000", 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.42e-06 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544417+0000", 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.529e-06 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544420+0000", 2024-09-16T13:01:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5620000000000002e-06 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592378+0000", 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047958354000000002 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592427+0000", 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9013e-05 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592478+0000", 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1034e-05 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240770 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.435254385, 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048189432999999997, 2024-09-16T13:01:34.025 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240770 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544422+0000", 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3200000000000001e-07 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544424+0000", 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4449999999999999e-06 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544427+0000", 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5049999999999999e-06 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544430+0000", 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.61e-06 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592511+0000", 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048080912000000003 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592563+0000", 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2107000000000001e-05 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592611+0000", 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8522000000000001e-05 2024-09-16T13:01:34.029 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240771 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 222208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.435219777999997, 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290128000000002, 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.030 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240771 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544457+0000", 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5700000000000002e-07 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544459+0000", 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6840000000000001e-06 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544466+0000", 2024-09-16T13:01:34.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999997e-06 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544469+0000", 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.45e-06 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592645+0000", 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048175216999999999 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592696+0000", 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1292999999999998e-05 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592747+0000", 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0751000000000002e-05 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.034 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240772 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.435196482999999, 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048403333999999999, 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240772 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544480+0000", 2024-09-16T13:01:34.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8900000000000001e-07 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544482+0000", 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8390000000000001e-06 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544486+0000", 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9369999999999997e-06 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544489+0000", 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8459999999999999e-06 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592779+0000", 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048290301000000001 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592831+0000", 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2335e-05 2024-09-16T13:01:34.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592883+0000", 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1786999999999997e-05 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240773 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 232448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.435185097999998, 2024-09-16T13:01:34.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048527963, 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240773 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544491+0000", 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544492+0000", 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.84e-07 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544493+0000", 2024-09-16T13:01:34.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6279999999999999e-06 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544498+0000", 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4689999999999999e-06 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544500+0000", 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592916+0000", 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048415672 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.592965+0000", 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9120999999999999e-05 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593019+0000", 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3950999999999999e-05 2024-09-16T13:01:34.043 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240774 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.435173442999996, 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048649546000000002, 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.044 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240774 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544503+0000", 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544505+0000", 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.61e-06 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544508+0000", 2024-09-16T13:01:34.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4929999999999998e-06 2024-09-16T13:01:34.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544511+0000", 2024-09-16T13:01:34.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4169999999999999e-06 2024-09-16T13:01:34.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593051+0000", 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048539604 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593103+0000", 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2068999999999999e-05 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593152+0000", 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9870999999999997e-05 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.048 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240775 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.435163189000001, 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048773328999999997, 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240775 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.544513+0000", 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545618+0000", 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011045810000000001 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545625+0000", 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6079999999999998e-06 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545644+0000", 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.823e-05 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545653+0000", 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3000000000000007e-06 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593186+0000", 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047533387000000003 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593235+0000", 2024-09-16T13:01:34.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8767000000000003e-05 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593286+0000", 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1456000000000003e-05 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240776 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:34.053 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.434011050999999, 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047754889000000002, 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240776 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:34.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545665+0000", 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545666+0000", 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0499999999999999e-06 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545671+0000", 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2270000000000004e-06 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545681+0000", 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0506000000000001e-05 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545687+0000", 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0970000000000002e-06 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593320+0000", 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047632999000000002 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593369+0000", 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8882999999999999e-05 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593420+0000", 2024-09-16T13:01:34.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1127e-05 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240777 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.43398243, 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047878318000000003, 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.058 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240777 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545694+0000", 2024-09-16T13:01:34.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545695+0000", 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545698+0000", 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0709999999999999e-06 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545707+0000", 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7629999999999995e-06 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545713+0000", 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2839999999999997e-06 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593453+0000", 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047740281000000002 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593518+0000", 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5493000000000005e-05 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593572+0000", 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3739000000000002e-05 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240778 4.8 4:1378be31:::rbd_data.11b55b44334a.000000000000011a:23 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.433957991, 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047999411999999998, 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240778 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545718+0000", 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545719+0000", 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.13e-07 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545728+0000", 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1560000000000001e-06 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545901+0000", 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017311599999999999 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.545909+0000", 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.028e-06 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593606+0000", 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047696660000000002 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593666+0000", 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9611999999999998e-05 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.593718+0000", 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2227e-05 2024-09-16T13:01:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:240951 4.8 4:10cc0d5f:::rbd_data.11b552946c0f.0000000000000120:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.425877722000003, 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.048226894999999999, 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240951 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553799+0000", 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2300000000000002e-07 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553801+0000", 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.686e-06 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553805+0000", 2024-09-16T13:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3039999999999998e-06 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.553809+0000", 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.599994+0000", 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046185154999999999 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.600112+0000", 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118589 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:39.602026+0000", 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019131370000000001 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:272626 4.12 4:4935e950:::rbd_data.11b59088eb24.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.521850100000002, 2024-09-16T13:01:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022439990000001, 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 272626 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457826+0000", 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457828+0000", 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0379999999999998e-06 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457835+0000", 2024-09-16T13:01:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7900000000000002e-06 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457853+0000", 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7671000000000001e-05 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457859+0000", 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2949999999999999e-06 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457913+0000", 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4112e-05 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:47.457922+0000", 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0850000000000003e-06 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.457879+0000", 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.99995641700000004 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458079+0000", 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020072300000000001 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.458304+0000", 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000224601 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459393+0000", 2024-09-16T13:01:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010886170000000001 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.459877+0000", 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048443500000000002 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460006+0000", 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012914 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:48.460070+0000", 2024-09-16T13:01:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4074999999999999e-05 2024-09-16T13:01:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.078 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:34.080 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T13:01:34.080 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:34.071+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:34.080 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:34.180 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:34.171+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:34.192 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:34.192 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:34.192 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:34.192 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:34.201 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:34.281 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:34.275+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:34.304 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:34.304 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:34.304 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:34.304 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:34.304 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:34.313 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:34.382 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:34.375+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:34.414 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.85951129, 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58474550199999997, 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549721+0000", 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549724+0000", 2024-09-16T13:01:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4370000000000001e-06 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549737+0000", 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3429e-05 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549763+0000", 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.641e-05 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549779+0000", 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5798999999999999e-05 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549880+0000", 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101111 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:45.549908+0000", 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7583e-05 2024-09-16T13:01:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131340+0000", 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58143166300000004 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.131866+0000", 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052601500000000001 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.132392+0000", 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052587100000000002 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134121+0000", 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017294190000000001 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134210+0000", 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9184999999999994e-05 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134374+0000", 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163602 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.134467+0000", 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2978000000000003e-05 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.03439424300001, 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013357803, 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374838+0000", 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374839+0000", 2024-09-16T13:01:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-07 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374845+0000", 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3900000000000001e-06 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374856+0000", 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1083e-05 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374871+0000", 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5237e-05 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383453+0000", 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085816580000000007 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383594+0000", 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014167699999999999 2024-09-16T13:01:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.383895+0000", 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030061699999999998 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388077+0000", 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041824469999999997 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388127+0000", 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9437000000000001e-05 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388168+0000", 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1298000000000003e-05 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388196+0000", 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7971999999999999e-05 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.03434698699999, 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014302726999999999, 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374886+0000", 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.374887+0000", 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.31e-06 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375013+0000", 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125831 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375068+0000", 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5624999999999997e-05 2024-09-16T13:01:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375078+0000", 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7829999999999993e-06 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384564+0000", 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094855059999999995 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384648+0000", 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4223999999999999e-05 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.384760+0000", 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111904 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388277+0000", 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035175050000000002 2024-09-16T13:01:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389105+0000", 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082753999999999998 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389158+0000", 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3016000000000002e-05 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389188+0000", 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0482999999999999e-05 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.034144051, 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014232572000000001, 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 48 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375088+0000", 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375090+0000", 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375095+0000", 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9139999999999996e-06 2024-09-16T13:01:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375105+0000", 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0022e-05 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375113+0000", 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6629999999999993e-06 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385770+0000", 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010657177 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385843+0000", 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.305e-05 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.385932+0000", 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9098999999999995e-05 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388968+0000", 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030355159999999998 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389233+0000", 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026533999999999998 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389294+0000", 2024-09-16T13:01:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0677999999999997e-05 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389321+0000", 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7056999999999999e-05 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.03411278900001, 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016861739000000001, 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 49 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375120+0000", 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1699999999999997e-07 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375126+0000", 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375141+0000", 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4387e-05 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375146+0000", 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6629999999999998e-06 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386381+0000", 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011234274000000001 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386444+0000", 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3743000000000003e-05 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386537+0000", 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2255000000000004e-05 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389031+0000", 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024947049999999998 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391889+0000", 2024-09-16T13:01:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028581510000000002 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391950+0000", 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1122000000000002e-05 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391981+0000", 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0969000000000001e-05 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.034077027, 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016932084999999999, 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 50 2024-09-16T13:01:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375155+0000", 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375156+0000", 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8400000000000003e-07 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375294+0000", 2024-09-16T13:01:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013767299999999999 2024-09-16T13:01:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375308+0000", 2024-09-16T13:01:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453999999999999e-05 2024-09-16T13:01:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375317+0000", 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4859999999999997e-06 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386878+0000", 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561599000000001 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.386952+0000", 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3687000000000001e-05 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387054+0000", 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101634 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389369+0000", 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002314787 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.391991+0000", 2024-09-16T13:01:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026225010000000002 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392043+0000", 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1437999999999998e-05 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392088+0000", 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5042e-05 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.03388239700001, 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832972000000002, 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 51 2024-09-16T13:01:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375350+0000", 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375351+0000", 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.043e-06 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375357+0000", 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375380+0000", 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3034999999999999e-05 2024-09-16T13:01:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375389+0000", 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4639999999999995e-06 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387590+0000", 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01220037 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387668+0000", 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8566999999999995e-05 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.387773+0000", 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010514900000000001 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.389822+0000", 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020489470000000002 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392101+0000", 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022786339999999999 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392155+0000", 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4320000000000002e-05 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392183+0000", 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.033835552, 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016887267000000001, 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 52 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375397+0000", 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375398+0000", 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.64e-07 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375403+0000", 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5400000000000003e-06 2024-09-16T13:01:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375413+0000", 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4490000000000005e-06 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.375419+0000", 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8200000000000002e-06 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388327+0000", 2024-09-16T13:01:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012907983 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388398+0000", 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1231999999999998e-05 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.388517+0000", 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118887 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.390334+0000", 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018167439999999999 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392194+0000", 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018606409999999999 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392259+0000", 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4391000000000001e-05 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.392284+0000", 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5616e-05 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 167.48955938099999, 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011427814, 2024-09-16T13:01:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919673+0000", 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919674+0000", 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2190000000000001e-06 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919683+0000", 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6000000000000007e-06 2024-09-16T13:01:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919701+0000", 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8519999999999999e-05 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919710+0000", 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2109999999999998e-06 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926017+0000", 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063074350000000001 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926115+0000", 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8280000000000001e-05 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.926399+0000", 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028394800000000001 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930097+0000", 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036976130000000002 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931023+0000", 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092600899999999999 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931065+0000", 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2005999999999997e-05 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931101+0000", 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5973000000000003e-05 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 167.489505474, 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011578616999999999, 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919727+0000", 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919728+0000", 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-06 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919737+0000", 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5890000000000003e-06 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919755+0000", 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8278999999999999e-05 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919761+0000", 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1260000000000003e-06 2024-09-16T13:01:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927222+0000", 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074610789999999998 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927446+0000", 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022415 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.927688+0000", 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241563 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931116+0000", 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034280579999999999 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931167+0000", 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0846000000000001e-05 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931219+0000", 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2617000000000001e-05 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931306+0000", 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6195999999999999e-05 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 167.489463169, 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013928605, 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919769+0000", 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919770+0000", 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000002e-07 2024-09-16T13:01:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919886+0000", 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115538 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919904+0000", 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958999999999999e-05 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919914+0000", 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9110000000000005e-06 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928828+0000", 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089142749999999993 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.928932+0000", 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103948 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.929063+0000", 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013156100000000001 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932149+0000", 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030853270000000001 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933593+0000", 2024-09-16T13:01:34.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014442019999999999 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933639+0000", 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5989000000000003e-05 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933698+0000", 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 167.48930474400001, 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013925875000000001, 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 73 2024-09-16T13:01:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919928+0000", 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919929+0000", 2024-09-16T13:01:34.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3430000000000001e-06 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919939+0000", 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7960000000000007e-06 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919955+0000", 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6322999999999999e-05 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919965+0000", 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8740000000000007e-06 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930156+0000", 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010191231 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930248+0000", 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1619000000000005e-05 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.930407+0000", 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158703 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933720+0000", 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033135949999999999 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933758+0000", 2024-09-16T13:01:34.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7489e-05 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933801+0000", 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3575999999999997e-05 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933854+0000", 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2326e-05 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 167.48925526299999, 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014139232, 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 74 2024-09-16T13:01:34.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919977+0000", 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919978+0000", 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.919985+0000", 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2200000000000003e-06 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920074+0000", 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8914999999999998e-05 2024-09-16T13:01:34.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920091+0000", 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6626e-05 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931363+0000", 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011272314 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931497+0000", 2024-09-16T13:01:34.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133356 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.931748+0000", 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025114400000000002 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.933874+0000", 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021265149999999998 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934015+0000", 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014105500000000001 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934076+0000", 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0863000000000003e-05 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934116+0000", 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0258e-05 2024-09-16T13:01:34.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 167.48912469300001, 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014859691, 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.499 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 75 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920108+0000", 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920110+0000", 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9400000000000001e-06 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920225+0000", 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114811 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920254+0000", 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9539e-05 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920269+0000", 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4943000000000001e-05 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932218+0000", 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011949402 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932376+0000", 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015774300000000001 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.932633+0000", 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025646600000000001 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934857+0000", 2024-09-16T13:01:34.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022241209999999999 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934868+0000", 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1294000000000001e-05 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934909+0000", 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0983999999999999e-05 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934968+0000", 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8448e-05 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 167.48894318000001, 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015501608, 2024-09-16T13:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 76 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920289+0000", 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920292+0000", 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1730000000000002e-06 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920299+0000", 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4599999999999997e-06 2024-09-16T13:01:34.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920320+0000", 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.920330+0000", 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934195+0000", 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013864978999999999 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934287+0000", 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2567999999999999e-05 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.934399+0000", 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111413 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935422+0000", 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001023524 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935700+0000", 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027801300000000002 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935740+0000", 2024-09-16T13:01:34.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9305999999999999e-05 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:46.935791+0000", 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1353000000000001e-05 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:34.511 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 166.678415513, 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011859606999999999, 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 93 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730817+0000", 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730818+0000", 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6300000000000004e-07 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730928+0000", 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110288 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730950+0000", 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2368999999999999e-05 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730957+0000", 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4640000000000003e-06 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737445+0000", 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064879839999999996 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737601+0000", 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015631299999999999 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.737811+0000", 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209564 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740427+0000", 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026164970000000002 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742596+0000", 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002169065 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742643+0000", 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6650000000000002e-05 2024-09-16T13:01:34.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742677+0000", 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3649999999999998e-05 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4210.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 166.67826655900001, 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014004180999999999, 2024-09-16T13:01:34.517 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4210", 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.136:0/3795352002", 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 94 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730966+0000", 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730967+0000", 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3300000000000001e-07 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730972+0000", 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2024-09-16T13:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730981+0000", 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3959999999999995e-06 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.730999+0000", 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.747e-05 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739733+0000", 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087346850000000007 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.739887+0000", 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015400200000000001 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0", 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.740032+0000", 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014465999999999999 2024-09-16T13:01:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.742097+0000", 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020648849999999998 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744794+0000", 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026973140000000001 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744904+0000", 2024-09-16T13:01:34.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109178 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T12:58:47.744970+0000", 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6625000000000007e-05 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64507 4.1e e83/31 4:7f7c7834:::rbd_data.11b5c2ee3542.0000000000000008:head v 83'225, mlcod=81'223)", 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:34.523 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.638853272000006, 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021284401000000001, 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770379+0000", 2024-09-16T13:01:34.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.770380+0000", 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8999999999999998e-07 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777141+0000", 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067617670000000001 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777157+0000", 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777167+0000", 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7999999999999993e-06 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777267+0000", 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100364 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777304+0000", 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7165000000000001e-05 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791601+0000", 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014297275999999999 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791664+0000", 2024-09-16T13:01:34.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2217000000000004e-05 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:64509 4.15 e83/31 4:ad2e18f4:::rbd_data.11b5c2ee3542.0000000000000009:head v 83'183, mlcod=81'180)", 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.632095370000002, 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014054395000000001, 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777137+0000", 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.777138+0000", 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3459999999999999e-06 2024-09-16T13:01:34.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778534+0000", 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013956769999999999 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778555+0000", 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1046000000000001e-05 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778564+0000", 2024-09-16T13:01:34.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9290000000000004e-06 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778633+0000", 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9363000000000007e-05 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.778666+0000", 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135000000000003e-05 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791145+0000", 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012479743 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:18.791192+0000", 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6156000000000003e-05 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.532 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4533.0:217754 4.17 e92/31 4:ee4048fa:::rbd_data.11b58362ce83.0000000000000002:head v 92'697, mlcod=91'693)", 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.909925315999999, 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011416822, 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499307+0000", 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499309+0000", 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6929999999999999e-06 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499322+0000", 2024-09-16T13:01:34.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2629999999999999e-05 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499354+0000", 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2135999999999999e-05 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499365+0000", 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111e-05 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499504+0000", 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013877599999999999 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.499565+0000", 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1939000000000004e-05 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510591+0000", 2024-09-16T13:01:34.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011025874999999999 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:33.510724+0000", 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013266299999999999 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.537 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:34.539 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:34.539 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:34.536+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:34.639 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:34.636+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:34.644 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:34.645 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:34.645 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:34.645 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:34.654 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:34.741 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:34.731+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:34.802 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:34.802 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:34.802 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:34.802 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:34.802 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:34.812 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:34.841 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:34.836+0000 7f9040d85700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 41070) UID: 0 2024-09-16T13:01:34.911 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:279128 4.1a 4.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-16T13:01:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.371305360999997, 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98766548899999995, 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 279128 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534646+0000", 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534648+0000", 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1459999999999999e-06 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534655+0000", 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5220000000000004e-06 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534674+0000", 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9717000000000001e-05 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534683+0000", 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6920000000000008e-06 2024-09-16T13:01:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534759+0000", 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6074000000000004e-05 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:55.534770+0000", 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0665999999999999e-05 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522281+0000", 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.98751079600000002 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.522312+0000", 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0876000000000001e-05 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.9 109/108 [PushOp(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29, version: 106'278, data_included: [0~688128,802816~10077,884736~295,1950271~32193], data_size: 730693, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:90536c5f:::rbd_data.11b5b95ccab.0000000000000003:29@106'278, size: 1982464, copy_subset: [0~1982464], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1982464, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.264286229, 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067875690000000002, 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641665+0000", 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.641666+0000", 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3060000000000001e-06 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644856+0000", 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00318995 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644894+0000", 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7382999999999998e-05 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.644913+0000", 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9415999999999998e-05 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645084+0000", 2024-09-16T13:01:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017036899999999999 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.645106+0000", 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2887e-05 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:00:56.648453+0000", 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003346258 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280527 4.13 4:cc227b3c:::rbd_data.11b5cb8797c6.0000000000000000:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.032826872000001, 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052047300000000003, 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280527 2024-09-16T13:01:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873125+0000", 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873126+0000", 2024-09-16T13:01:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6530000000000001e-06 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873133+0000", 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0940000000000002e-06 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873153+0000", 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9538000000000002e-05 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873161+0000", 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1009999999999992e-06 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873245+0000", 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4075999999999999e-05 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873360+0000", 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114951 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873530+0000", 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016994699999999999 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.877617+0000", 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040873000000000003 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878160+0000", 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054299199999999995 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878251+0000", 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1062000000000003e-05 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878329+0000", 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8016000000000007e-05 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280530 4.18 4:1e233aba:::rbd_data.11b5cb8797c6.0000000000000003:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.032764868000001, 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056839330000000004, 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280530 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873187+0000", 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8199999999999999e-07 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873191+0000", 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4599999999999999e-06 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873198+0000", 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8399999999999997e-06 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873201+0000", 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.641e-06 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873298+0000", 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6682999999999996e-05 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873451+0000", 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015304000000000001 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.873692+0000", 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024124300000000001 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.876380+0000", 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026873980000000001 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878771+0000", 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002391013 2024-09-16T13:01:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878824+0000", 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3136000000000001e-05 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:05.878870+0000", 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6796999999999999e-05 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280650 4.1b 4:d8560506:::rbd_header.11b5de89dd5b:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.146440458000001, 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085055370000000005, 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280650 2024-09-16T13:01:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759511+0000", 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759514+0000", 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7429999999999998e-06 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759524+0000", 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8309999999999996e-06 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759540+0000", 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6506000000000001e-05 2024-09-16T13:01:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.759548+0000", 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1810000000000001e-06 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.761995+0000", 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002446518 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.762120+0000", 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125124 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.762945+0000", 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082531599999999998 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.766013+0000", 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030673549999999999 2024-09-16T13:01:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.767864+0000", 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001851161 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.767939+0000", 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5012000000000006e-05 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.768016+0000", 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7789999999999999e-05 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280740 4.18 4:1df486ae:::rbd_data.11b5de89dd5b.0000000000000000:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.996948328999999, 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070806940000000002, 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280740 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909003+0000", 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909005+0000", 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.317e-06 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909016+0000", 2024-09-16T13:01:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0852999999999999e-05 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909052+0000", 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5256000000000003e-05 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909064+0000", 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2413e-05 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909168+0000", 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103615 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909335+0000", 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016715100000000001 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909599+0000", 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000264391 2024-09-16T13:01:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.914767+0000", 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051679309999999997 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915839+0000", 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001072415 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915986+0000", 2024-09-16T13:01:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014609099999999999 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.916084+0000", 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8261e-05 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:280741 4.13 4:ce26f199:::rbd_data.11b5de89dd5b.0000000000000001:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T13:01:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.996861741, 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062230289999999997, 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280741 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909090+0000", 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909091+0000", 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0100000000000001e-06 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909096+0000", 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4399999999999996e-06 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909107+0000", 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.066e-05 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909113+0000", 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9460000000000003e-06 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909206+0000", 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3450999999999997e-05 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909341+0000", 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135267 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6", 2024-09-16T13:01:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.909519+0000", 2024-09-16T13:01:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177603 2024-09-16T13:01:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.913993+0000", 2024-09-16T13:01:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044734989999999997 2024-09-16T13:01:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915108+0000", 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001115001 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915207+0000", 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9562999999999995e-05 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:06.915313+0000", 2024-09-16T13:01:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010558899999999999 2024-09-16T13:01:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281211 4.5 4:a14f106d:::rbd_header.11b515823229:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.108314728, 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093462500000000004, 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281211 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797637+0000", 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797641+0000", 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4259999999999996e-06 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797662+0000", 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.069e-05 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797705+0000", 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3495999999999998e-05 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797719+0000", 2024-09-16T13:01:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4003e-05 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797828+0000", 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010857 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.797840+0000", 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.235e-05 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.802951+0000", 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051111079999999996 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.803147+0000", 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000196109 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806806+0000", 2024-09-16T13:01:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003658246 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806865+0000", 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9246999999999997e-05 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.806983+0000", 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011800500000000001 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281308 4.1b 4:db2b1562:::rbd_data.11b515823229.000000000000002f:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.981415717000001, 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056950619999999999, 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281308 2024-09-16T13:01:34.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924536+0000", 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7999999999999995e-07 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924539+0000", 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9799999999999998e-06 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924546+0000", 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4620000000000001e-06 2024-09-16T13:01:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924550+0000", 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0040000000000001e-06 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924628+0000", 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7749000000000004e-05 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.924853+0000", 2024-09-16T13:01:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000225491 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.925042+0000", 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018871099999999999 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.928196+0000", 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031538120000000002 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930092+0000", 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001896556 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930167+0000", 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5111999999999995e-05 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.930231+0000", 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3504999999999996e-05 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:281318 4.e 4:7158d438:::rbd_data.11b515823229.0000000000000081:head [delete] snapc 1e=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.976529816999999, 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058318850000000002, 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 281318 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929422+0000", 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929423+0000", 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2139999999999999e-06 2024-09-16T13:01:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929428+0000", 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.826e-06 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929439+0000", 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1549000000000001e-05 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929444+0000", 2024-09-16T13:01:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0050000000000004e-06 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929506+0000", 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1927999999999994e-05 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.929632+0000", 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012569299999999999 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935090+0000", 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054580920000000003 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935154+0000", 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3688000000000004e-05 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:10.935253+0000", 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9889999999999994e-05 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282172 4.13 4:c99eeb3d:::rbd_data.11b5b3dc7343.0000000000000001:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.942314+0000", 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.963637628000001, 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005440907, 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282172 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942314+0000", 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942314+0000", 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942316+0000", 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.176e-06 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942326+0000", 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0514e-05 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942361+0000", 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4875000000000001e-05 2024-09-16T13:01:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942374+0000", 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2924e-05 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942515+0000", 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140746 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.943277+0000", 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076154200000000003 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947497+0000", 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042207939999999999 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947593+0000", 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5166000000000003e-05 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947755+0000", 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016217 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282176 4.1 4:82eb3962:::rbd_data.11b5b3dc7343.0000000000000005:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.942450+0000", 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.963501530999999, 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051970929999999999, 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282176 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942450+0000", 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942450+0000", 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942451+0000", 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1230000000000001e-06 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942457+0000", 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8370000000000002e-06 2024-09-16T13:01:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942474+0000", 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6663e-05 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942484+0000", 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0462000000000001e-05 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.942618+0000", 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134167 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.943112+0000", 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049358500000000005 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 5", 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.943411+0000", 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000299296 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947145+0000", 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037339589999999998 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947437+0000", 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029157299999999999 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947547+0000", 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110287 2024-09-16T13:01:34.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947647+0000", 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100141 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282181 4.d 4:b2f030fd:::rbd_data.11b5b3dc7343.0000000000000034:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:15.946061+0000", 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.959890746999999, 2024-09-16T13:01:34.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066091789999999997, 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282181 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946061+0000", 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946061+0000", 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946064+0000", 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0359999999999998e-06 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946076+0000", 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2614e-05 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946100+0000", 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3198999999999999e-05 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946115+0000", 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5953000000000001e-05 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946312+0000", 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019610499999999999 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.946957+0000", 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00064557500000000001 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2", 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.947239+0000", 2024-09-16T13:01:34.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028224599999999999 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.949960+0000", 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027203399999999999 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952517+0000", 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025567709999999998 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952601+0000", 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4557999999999999e-05 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:15.952670+0000", 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8782000000000007e-05 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:282774 4.c 4:36b2d631:::rbd_object_map.11b51a3eb68b:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:19.869669+0000", 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.036282635999999, 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055610959999999997, 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282774 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.988 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869669+0000", 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869669+0000", 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869671+0000", 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6240000000000002e-06 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869684+0000", 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2152e-05 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869715+0000", 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1659000000000003e-05 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869729+0000", 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3563e-05 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869823+0000", 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4333999999999995e-05 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.869831+0000", 2024-09-16T13:01:34.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5859999999999999e-06 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.872858+0000", 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030270700000000002 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.873002+0000", 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014394599999999999 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.873214+0000", 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021210200000000001 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875001+0000", 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001786852 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875103+0000", 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102058 2024-09-16T13:01:34.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875170+0000", 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7019000000000003e-05 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:19.875230+0000", 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0130999999999997e-05 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:283012 4.9 4:9572e2af:::rbd_data.11b5c2c64266.0000000000000005:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:20.979407+0000", 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.926544617999999, 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070250620000000003, 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 283012 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:34.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979407+0000", 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979407+0000", 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:34.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979408+0000", 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.514e-06 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979416+0000", 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2830000000000004e-06 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979438+0000", 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2218999999999999e-05 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979450+0000", 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2303000000000001e-05 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.979580+0000", 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013011699999999999 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.980107+0000", 2024-09-16T13:01:34.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052712500000000003 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.980398+0000", 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029023299999999998 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.984244+0000", 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038468450000000002 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:34.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.986286+0000", 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020413279999999998 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.986384+0000", 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8482999999999999e-05 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:20.986432+0000", 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7611999999999997e-05 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.000 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:283597 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138)", 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:24.930684+0000", 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.9752669580000006, 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013228014, 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:35.001 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 283597 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.930684+0000", 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.930684+0000", 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.930688+0000", 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3390000000000001e-06 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.930702+0000", 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4637000000000001e-05 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.943493+0000", 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012790625 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.943521+0000", 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7787e-05 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.943608+0000", 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7653999999999998e-05 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.943809+0000", 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000200991 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:24.943912+0000", 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102981 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:283779 4.13 4:ccb453e7:::rbd_header.11b55c5d4be2:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:25.909652+0000", 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.9962996260000008, 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070661450000000002, 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 283779 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909652+0000", 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909652+0000", 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909654+0000", 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2730000000000001e-06 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909665+0000", 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0478e-05 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909690+0000", 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5074000000000001e-05 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909698+0000", 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8130000000000002e-06 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909812+0000", 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011328999999999999 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.909824+0000", 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2573e-05 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.914045+0000", 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042204269999999997 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.914291+0000", 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024655100000000002 2024-09-16T13:01:35.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.916385+0000", 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002093953 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.916514+0000", 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012908400000000001 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:25.916718+0000", 2024-09-16T13:01:35.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020362900000000001 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:284231 4.1b 4:decce19f:::rbd_header.11b58f77e1c4:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:28.934053+0000", 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.9718986909999998, 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010865763000000001, 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:35.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 284231 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.934053+0000", 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.934053+0000", 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.934057+0000", 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7799999999999998e-06 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.934074+0000", 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7353000000000001e-05 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.934142+0000", 2024-09-16T13:01:35.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8634999999999996e-05 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.934163+0000", 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0293e-05 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.934295+0000", 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013240600000000001 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.934305+0000", 2024-09-16T13:01:35.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4930000000000004e-06 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.942355+0000", 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080499070000000002 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.942440+0000", 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5734000000000003e-05 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:35.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.942856+0000", 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041529300000000001 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.944279+0000", 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014236279999999999 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.944783+0000", 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050361899999999996 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.944858+0000", 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4867e-05 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:28.944918+0000", 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0754999999999997e-05 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:284661 4.1a 4:5ce9034b:::rbd_data.11b5b1b9bb41.000000000000000e:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-16T13:01:35.017 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:31.090776+0000", 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8151758550000001, 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005875323, 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 284661 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.090776+0000", 2024-09-16T13:01:35.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.090776+0000", 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.090779+0000", 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0570000000000001e-06 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.090793+0000", 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4048e-05 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.090827+0000", 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4344999999999997e-05 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.090839+0000", 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2326e-05 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.090982+0000", 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014285900000000001 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.091522+0000", 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054018999999999996 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3", 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.091822+0000", 2024-09-16T13:01:35.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029957900000000001 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.093730+0000", 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00190782 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.096423+0000", 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026933870000000002 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.096537+0000", 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114261 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:31.096651+0000", 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113451 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:285025 4.9 4:94f499f8:::rbd_object_map.11b5c378985b:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-16T13:01:35.023 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:33.980161+0000", 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.92578996800000002, 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071495899999999999, 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 285025 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.980161+0000", 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.980161+0000", 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.980163+0000", 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8169999999999999e-06 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.980172+0000", 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8740000000000001e-06 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.980196+0000", 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3811999999999998e-05 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.980204+0000", 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0360000000000006e-06 2024-09-16T13:01:35.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.980275+0000", 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1298999999999996e-05 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.980284+0000", 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3890000000000005e-06 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.984081+0000", 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037970930000000001 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.984195+0000", 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114071 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.984365+0000", 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017064399999999999 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.986915+0000", 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025498140000000001 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.987168+0000", 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025295 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.987259+0000", 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0910999999999997e-05 2024-09-16T13:01:35.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:33.987311+0000", 2024-09-16T13:01:35.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1879999999999998e-05 2024-09-16T13:01:35.030 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.030 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.030 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.030 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.030 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.030 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:35.031 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T13:01:35.032 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T13:01:35.033 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:01:35.033 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:35.028+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:35.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T13:01:35.133 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:35.123+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:35.187 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:35.187 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:35.187 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-16T13:01:35.187 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:35.196 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:01:35.234 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:35.228+0000 7f9040d85700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 41070) UID: 0 2024-09-16T13:01:35.291 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:35.291 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:35.291 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:35.291 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:35.291 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:35.300 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:01:35.335 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:35.332+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:35.404 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:35.404 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:35.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:35.404 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:35.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:285083 4.5 4:a7daa15d:::rbd_id.image_client.0-clone3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:34.059223+0000", 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3402872139999999, 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.92859871100000002, 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 285083 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059223+0000", 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059223+0000", 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059225+0000", 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0980000000000001e-06 2024-09-16T13:01:35.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059232+0000", 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0450000000000003e-06 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059249+0000", 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7566000000000001e-05 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059260+0000", 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.048e-05 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059386+0000", 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012567599999999999 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059404+0000", 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8771e-05 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.986070+0000", 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.926665238 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.986214+0000", 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000144318 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.986311+0000", 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6794999999999997e-05 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.987271+0000", 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096042999999999996 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.987706+0000", 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043457400000000002 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.987745+0000", 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9002999999999999e-05 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.987821+0000", 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6717000000000004e-05 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.18 148/147 [PushOp(4:18f89e6a:::rbd_data.11b571860f2a.0000000000000001:head, version: 137'404, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18f89e6a:::rbd_data.11b571860f2a.0000000000000001:head@137'404, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.014538+0000", 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.384972121, 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021845160000000001, 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014538+0000", 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014538+0000", 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014539+0000", 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8199999999999998e-07 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014556+0000", 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7802e-05 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014578+0000", 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1526999999999999e-05 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014588+0000", 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8810000000000008e-06 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014646+0000", 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8542000000000003e-05 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014654+0000", 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7840000000000004e-06 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.016722+0000", 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002068098 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGRecoveryDelete(4.18 e148,147 4:19cffc52:::rbd_data.11b5b1b9bb41.000000000000000f:head,144'405)", 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.017888+0000", 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.381622026, 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021382929999999999, 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017888+0000", 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017888+0000", 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017888+0000", 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9800000000000003e-07 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017893+0000", 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4580000000000003e-06 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017911+0000", 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7646999999999999e-05 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017917+0000", 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.849e-06 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017951+0000", 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4029999999999998e-05 2024-09-16T13:01:35.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017956+0000", 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6949999999999999e-06 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020026+0000", 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002070016 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:285115 4.a 4:51b44bfb:::rbd_data.11b5c378985b.0000000000000003:head [delete] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.020871+0000", 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.37863867600000001, 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040661120000000002, 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 285115 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.419 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020871+0000", 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020871+0000", 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020873+0000", 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9769999999999999e-06 2024-09-16T13:01:35.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020880+0000", 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3949999999999998e-06 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020893+0000", 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3852999999999999e-05 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020908+0000", 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4944e-05 2024-09-16T13:01:35.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020961+0000", 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3028000000000001e-05 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.021694+0000", 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073269199999999996 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.021835+0000", 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014123599999999999 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.023554+0000", 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017183770000000001 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.024730+0000", 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001176163 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.024784+0000", 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4014000000000002e-05 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.024937+0000", 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015343299999999999 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:596 4.18 e148/147 4:18f89e6a:::rbd_data.11b571860f2a.0000000000000001:11 v 148'407, mlcod=137'404)", 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.030648+0000", 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.36886155300000001, 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027864209999999999, 2024-09-16T13:01:35.424 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030648+0000", 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030648+0000", 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030649+0000", 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9699999999999995e-07 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030656+0000", 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5939999999999999e-06 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030683+0000", 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7506000000000001e-05 2024-09-16T13:01:35.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030689+0000", 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0730000000000001e-06 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030729+0000", 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0185999999999998e-05 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030752+0000", 2024-09-16T13:01:35.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3037e-05 2024-09-16T13:01:35.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.033385+0000", 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026328829999999999 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.033435+0000", 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9444999999999998e-05 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:597 4.18 e148/147 4:18688110:::rbd_data.11b5b635ee69.00000000000000da:17 v 148'409, mlcod=144'405)", 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.039138+0000", 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.360372368, 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00209553, 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039138+0000", 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039138+0000", 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039138+0000", 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4000000000000001e-07 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039145+0000", 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3740000000000003e-06 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039189+0000", 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4141000000000003e-05 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039195+0000", 2024-09-16T13:01:35.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4489999999999996e-06 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039242+0000", 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7117000000000003e-05 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039275+0000", 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.269e-05 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.041191+0000", 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019155470000000001 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.041233+0000", 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2571999999999998e-05 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.434 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:602 4.18 e148/147 4:18b02557:::rbd_data.11b54010928d.00000000000000aa:1b v 148'417, mlcod=148'413)", 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.049605+0000", 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.349904777, 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025643799999999998, 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049605+0000", 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049605+0000", 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049606+0000", 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2799999999999995e-07 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049612+0000", 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.409e-06 2024-09-16T13:01:35.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049630+0000", 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7592999999999999e-05 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049635+0000", 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6870000000000002e-06 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050022+0000", 2024-09-16T13:01:35.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038735799999999997 2024-09-16T13:01:35.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050040+0000", 2024-09-16T13:01:35.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8542999999999998e-05 2024-09-16T13:01:35.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.052129+0000", 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020881509999999999 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.052170+0000", 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0911000000000002e-05 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:603 4.18 e148/147 4:1a8b08a8:::rbd_data.11b54010928d.0000000000000138:1b v 148'419, mlcod=148'413)", 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.050495+0000", 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.34901488600000002, 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023761250000000002, 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050495+0000", 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050495+0000", 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050495+0000", 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050499+0000", 2024-09-16T13:01:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.709e-06 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050505+0000", 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7660000000000004e-06 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050508+0000", 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1209999999999998e-06 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051351+0000", 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084336899999999995 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051371+0000", 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9389e-05 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.052832+0000", 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014615710000000001 2024-09-16T13:01:35.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.052871+0000", 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8832999999999999e-05 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(osd.6.0:11 4.1f e148/147 ondisk, result = 0)", 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.051877+0000", 2024-09-16T13:01:35.444 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.34763335299999998, 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026759090000000002, 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051877+0000", 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051877+0000", 2024-09-16T13:01:35.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051877+0000", 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1500000000000004e-07 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051882+0000", 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3939999999999998e-06 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051893+0000", 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0906e-05 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051898+0000", 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1989999999999997e-06 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.053782+0000", 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001884265 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.054490+0000", 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070751400000000002 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.054552+0000", 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2916000000000005e-05 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:605 4.18 e148/147 4:1c9f5a62:::rbd_data.11b515823229.00000000000000ff:1e v 148'423, mlcod=148'419)", 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.059635+0000", 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.33987474499999998, 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030075190000000002, 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059635+0000", 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059635+0000", 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059636+0000", 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059641+0000", 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6450000000000002e-06 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059651+0000", 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6609999999999995e-06 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059656+0000", 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059728+0000", 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1700999999999997e-05 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059756+0000", 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8459999999999999e-05 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.062610+0000", 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028539809999999998 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.062643+0000", 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2407999999999999e-05 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:606 4.18 e148/147 4:19e0f116:::rbd_data.11b515823229.00000000000000c7:1e v 148'425, mlcod=148'419)", 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.060608+0000", 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.33890183299999999, 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002967969, 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.453 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060608+0000", 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060608+0000", 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060609+0000", 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5199999999999997e-07 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060615+0000", 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060636+0000", 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0913000000000001e-05 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060648+0000", 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1732e-05 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060796+0000", 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014830099999999999 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060814+0000", 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8168999999999998e-05 2024-09-16T13:01:35.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.063528+0000", 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027139989999999999 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.063576+0000", 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8016999999999998e-05 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.457 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:285148 4.a 4:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e148)", 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.069603+0000", 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.32990725799999998, 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035980309999999998, 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 285148 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069603+0000", 2024-09-16T13:01:35.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069603+0000", 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069605+0000", 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.968e-06 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069611+0000", 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.229e-06 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069623+0000", 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1779e-05 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069630+0000", 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9820000000000002e-06 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069754+0000", 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012427299999999999 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069942+0000", 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000188097 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.072929+0000", 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029868579999999998 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.073032+0000", 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010353699999999999 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.073201+0000", 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016830799999999999 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.a 148/147 [PushOp(4:535385f0:::rbd_children:head, version: 146'1349, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 1, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:535385f0:::rbd_children:head@146'1349, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.070394+0000", 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.32911588200000003, 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020175060000000001, 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070394+0000", 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070394+0000", 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070395+0000", 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3799999999999996e-07 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070413+0000", 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7934000000000001e-05 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070429+0000", 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6325999999999999e-05 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070439+0000", 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5279999999999992e-06 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070527+0000", 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8209999999999997e-05 2024-09-16T13:01:35.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070535+0000", 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5990000000000002e-06 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.072412+0000", 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001876171 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:611 4.13 e148/147 4:caad81d8:::rbd_data.11b5b635ee69.00000000000001c5:17 v 148'490, mlcod=148'486)", 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.078727+0000", 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.320782819, 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022223099999999999, 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078727+0000", 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078727+0000", 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078728+0000", 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3200000000000005e-07 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078735+0000", 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9820000000000002e-06 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078743+0000", 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1750000000000005e-06 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078748+0000", 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4530000000000001e-06 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078889+0000", 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014086799999999999 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078908+0000", 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8797999999999999e-05 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.080921+0000", 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002013239 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.080949+0000", 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8163000000000001e-05 2024-09-16T13:01:35.470 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:612 4.13 e148/147 4:cfc5f29c:::rbd_data.11b5b635ee69.00000000000001aa:17 v 148'492, mlcod=148'486)", 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.079400+0000", 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.32010948700000003, 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021213320000000001, 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079400+0000", 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079400+0000", 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079401+0000", 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6700000000000003e-07 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079407+0000", 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9920000000000004e-06 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079416+0000", 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9609999999999994e-06 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079421+0000", 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2050000000000001e-06 2024-09-16T13:01:35.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079600+0000", 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017862999999999999 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079619+0000", 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8817e-05 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.081487+0000", 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001868179 2024-09-16T13:01:35.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.081522+0000", 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4881e-05 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:613 4.13 e148/147 4:c99eeb3d:::rbd_data.11b5b3dc7343.0000000000000001:19 v 148'494, mlcod=148'490)", 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.084737+0000", 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.31477306900000002, 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00215766, 2024-09-16T13:01:35.475 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084737+0000", 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084737+0000", 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084738+0000", 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084744+0000", 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.917e-06 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084787+0000", 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2045000000000001e-05 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084792+0000", 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9549999999999997e-06 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084847+0000", 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084868+0000", 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0616999999999999e-05 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.086859+0000", 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019913359999999998 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.086895+0000", 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5581999999999999e-05 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:617 4.13 e148/147 4:cba8a5ab:::rbd_data.11b5403f243a.0000000000000036:1f v 148'500, mlcod=148'494)", 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.096844+0000", 2024-09-16T13:01:35.479 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.30266546799999999, 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026079720000000001, 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096844+0000", 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096844+0000", 2024-09-16T13:01:35.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096851+0000", 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0569999999999998e-06 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096855+0000", 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6970000000000001e-06 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096862+0000", 2024-09-16T13:01:35.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0339999999999999e-06 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096873+0000", 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0902e-05 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096911+0000", 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8331999999999998e-05 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096930+0000", 2024-09-16T13:01:35.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8403e-05 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.099411+0000", 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024809340000000002 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.099452+0000", 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1613000000000003e-05 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:625 4.e e148/147 4:76368bb3:::rbd_data.11b54010928d.0000000000000005:1b v 148'837, mlcod=148'833)", 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.128477+0000", 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.27103249000000001, 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021061410000000002, 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128477+0000", 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128477+0000", 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128478+0000", 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1399999999999996e-07 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128485+0000", 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5339999999999996e-06 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128497+0000", 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2311e-05 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128503+0000", 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9279999999999998e-06 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128555+0000", 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1858999999999999e-05 2024-09-16T13:01:35.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128585+0000", 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0125e-05 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.130534+0000", 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001948731 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.130584+0000", 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9839000000000002e-05 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:629 4.e e148/147 4:76f7a092:::rbd_data.11b5403f243a.0000000000000102:1f v 148'845, mlcod=148'841)", 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.139198+0000", 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.26031217099999998, 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003046506, 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.488 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139198+0000", 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139198+0000", 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139199+0000", 2024-09-16T13:01:35.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9299999999999996e-07 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139205+0000", 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5050000000000004e-06 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139219+0000", 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3356999999999999e-05 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139224+0000", 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8640000000000001e-06 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139260+0000", 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5886999999999997e-05 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139280+0000", 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9891999999999999e-05 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.142198+0000", 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029173749999999998 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.142244+0000", 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6733000000000001e-05 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:630 4.e e148/147 4:750dfe80:::rbd_data.11b5403f243a.0000000000000107:1f v 148'847, mlcod=148'841)", 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.140096+0000", 2024-09-16T13:01:35.492 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.25941441700000001, 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024568929999999999, 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140096+0000", 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140096+0000", 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:35.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140096+0000", 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8899999999999998e-07 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140102+0000", 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6960000000000002e-06 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140114+0000", 2024-09-16T13:01:35.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1973e-05 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140120+0000", 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.57e-06 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140159+0000", 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8958999999999997e-05 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140177+0000", 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8811e-05 2024-09-16T13:01:35.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.142521+0000", 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00234323 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.142552+0000", 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1764999999999998e-05 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:35.497 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:35.497 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:35.498 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T13:01:35.499 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:35.496+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:35.599 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:35.591+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:35.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T13:01:35.604 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-16T13:01:35.604 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:35.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T13:01:35.699 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:35.691+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:35.708 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:35.709 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-16T13:01:35.709 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-16T13:01:35.709 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-16T13:01:35.709 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:35.718 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T13:01:35.737 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:35.756 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:35.774 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:35.792 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:35.800 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:35.796+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:35.814 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:35.836 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:35.836 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:35.841 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:35.846 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T13:01:35.865 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:01:35.890 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:35.892 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:35.892 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:35.892 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:35.892 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:35.892 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:35.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:35.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-16T13:01:35.893 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:35.901 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:35.896+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:35.969 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:35.969 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:35.978 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T13:01:36.001 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-16T13:01:36.002 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:35.995+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:36.006 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T13:01:36.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T13:01:36.094 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:36.094 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:36.103 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:36.100+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:36.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T13:01:36.201 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:36.202 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:36.204 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:36.200+0000 7f9040d85700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 41070) UID: 0 2024-09-16T13:01:36.212 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T13:01:36.306 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:36.300+0000 7f9040d85700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 41070) UID: 0 2024-09-16T13:01:36.341 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:36.341 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:36.349 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T13:01:36.406 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-16T13:01:36.399+0000 7f351e8cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22172) UID: 0 2024-09-16T13:01:36.447 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:36.448 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:36.457 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T13:01:36.507 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:36.504+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:36.558 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:36.558 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:36.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T13:01:36.608 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:36.604+0000 7f9040d85700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 41070) UID: 0 2024-09-16T13:01:36.667 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:36.667 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:36.676 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T13:01:36.709 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:36.703+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:36.782 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:36.782 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:36.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T13:01:36.810 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:36.804+0000 7f9040d85700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 41070) UID: 0 2024-09-16T13:01:36.893 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:36.893 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:36.903 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T13:01:36.947 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:36.944+0000 7f9040d85700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 41070) UID: 0 2024-09-16T13:01:37.010 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:37.008+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:37.020 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T13:01:37.029 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:37.029 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:37.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T13:01:37.040 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T13:01:37.112 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:37.108+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:37.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:37.139 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:37.149 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T13:01:37.212 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-09-16T13:01:37.208+0000 7fa4acb9f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22335) UID: 0 2024-09-16T13:01:37.293 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:37.293 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:37.301 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T13:01:37.351 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:37.348+0000 7f9040d85700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 41070) UID: 0 2024-09-16T13:01:37.400 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:37.400 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:37.409 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T13:01:37.414 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:37.408+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:37.507 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:37.507 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:37.515 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T13:01:37.516 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:37.512+0000 7f9040d85700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 41070) UID: 0 2024-09-16T13:01:37.617 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-16T13:01:37.607+0000 7fb9bb62a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22178) UID: 0 2024-09-16T13:01:37.665 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:37.665 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:37.674 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T13:01:37.718 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-16T13:01:37.715+0000 7fbb9e4df700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22174) UID: 0 2024-09-16T13:01:37.770 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:37.771 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:37.779 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T13:01:37.819 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-09-16T13:01:37.816+0000 7f9040d85700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 41070) UID: 0 2024-09-16T13:01:37.880 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:37.880 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:37.889 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T13:01:37.904 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T13:01:37.931 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-09-16T13:01:37.928+0000 7f466869f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33592) UID: 0 2024-09-16T13:01:38.017 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T13:01:38.020 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-16T13:01:38.011+0000 7f31d03e6700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22175) UID: 0 2024-09-16T13:01:38.027 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T13:01:38.038 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:38.038 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:38.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T13:01:38.048 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T13:01:38.059 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-16T13:01:38.083 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-16T13:01:38.101 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-16T13:01:38.120 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-16T13:01:38.116+0000 7f192cd37700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22332) UID: 0 2024-09-16T13:01:38.122 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-16T13:01:38.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-16T13:01:38.146 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T13:01:38.146 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-16T13:01:38.156 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T13:01:38.158 INFO:teuthology.orchestra.run.smithi042.stdout:All operations completed A-OK! 2024-09-16T13:01:38.170 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-16T13:01:38.191 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-16T13:01:38.195 DEBUG:teuthology.parallel:result is None 2024-09-16T13:01:38.196 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-16T13:01:38.205 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-16T13:01:38.213 INFO:tasks.thrashosds:joining thrashosds 2024-09-16T13:01:38.214 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-16T13:01:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T13:01:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T13:01:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T13:01:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T13:01:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T13:01:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T13:01:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T13:01:38.214 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T13:01:38.251 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-16T13:01:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-16T13:01:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-16T13:01:38.252 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:38.260 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T13:01:38.359 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-16T13:01:38.359 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:285083 4.5 4:a7daa15d:::rbd_id.image_client.0-clone3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:34.059223+0000", 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.2958498800000005, 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.92859871100000002, 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 285083 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059223+0000", 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059223+0000", 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059225+0000", 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0980000000000001e-06 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059232+0000", 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0450000000000003e-06 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059249+0000", 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7566000000000001e-05 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059260+0000", 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.048e-05 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059386+0000", 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012567599999999999 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-16T13:01:38.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.059404+0000", 2024-09-16T13:01:38.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8771e-05 2024-09-16T13:01:38.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.986070+0000", 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.926665238 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.986214+0000", 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000144318 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4", 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.986311+0000", 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6794999999999997e-05 2024-09-16T13:01:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.987271+0000", 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096042999999999996 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.987706+0000", 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043457400000000002 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.987745+0000", 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9002999999999999e-05 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:34.987821+0000", 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6717000000000004e-05 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.18 148/147 [PushOp(4:18f89e6a:::rbd_data.11b571860f2a.0000000000000001:head, version: 137'404, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18f89e6a:::rbd_data.11b571860f2a.0000000000000001:head@137'404, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.014538+0000", 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3405347870000002, 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021845160000000001, 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014538+0000", 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014538+0000", 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014539+0000", 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8199999999999998e-07 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014556+0000", 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7802e-05 2024-09-16T13:01:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014578+0000", 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1526999999999999e-05 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014588+0000", 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8810000000000008e-06 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014646+0000", 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8542000000000003e-05 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.014654+0000", 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7840000000000004e-06 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.016722+0000", 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002068098 2024-09-16T13:01:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGRecoveryDelete(4.18 e148,147 4:19cffc52:::rbd_data.11b5b1b9bb41.000000000000000f:head,144'405)", 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.017888+0000", 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3371846920000001, 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021382929999999999, 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017888+0000", 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017888+0000", 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017888+0000", 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9800000000000003e-07 2024-09-16T13:01:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017893+0000", 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4580000000000003e-06 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017911+0000", 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7646999999999999e-05 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017917+0000", 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.849e-06 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017951+0000", 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4029999999999998e-05 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.017956+0000", 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6949999999999999e-06 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020026+0000", 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002070016 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:285115 4.a 4:51b44bfb:::rbd_data.11b5c378985b.0000000000000003:head [delete] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.020871+0000", 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3342013420000001, 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040661120000000002, 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 285115 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020871+0000", 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020871+0000", 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020873+0000", 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9769999999999999e-06 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020880+0000", 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3949999999999998e-06 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020893+0000", 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3852999999999999e-05 2024-09-16T13:01:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020908+0000", 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4944e-05 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.020961+0000", 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3028000000000001e-05 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.021694+0000", 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073269199999999996 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 1", 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.021835+0000", 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014123599999999999 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.023554+0000", 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017183770000000001 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.024730+0000", 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001176163 2024-09-16T13:01:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.024784+0000", 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4014000000000002e-05 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.024937+0000", 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015343299999999999 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:596 4.18 e148/147 4:18f89e6a:::rbd_data.11b571860f2a.0000000000000001:11 v 148'407, mlcod=137'404)", 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.030648+0000", 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.324424219, 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027864209999999999, 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030648+0000", 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030648+0000", 2024-09-16T13:01:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030649+0000", 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9699999999999995e-07 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030656+0000", 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5939999999999999e-06 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030683+0000", 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7506000000000001e-05 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030689+0000", 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0730000000000001e-06 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030729+0000", 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0185999999999998e-05 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.030752+0000", 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3037e-05 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.033385+0000", 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026328829999999999 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.033435+0000", 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9444999999999998e-05 2024-09-16T13:01:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:597 4.18 e148/147 4:18688110:::rbd_data.11b5b635ee69.00000000000000da:17 v 148'409, mlcod=144'405)", 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.039138+0000", 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3159350339999998, 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00209553, 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039138+0000", 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039138+0000", 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039138+0000", 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4000000000000001e-07 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039145+0000", 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3740000000000003e-06 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039189+0000", 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4141000000000003e-05 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039195+0000", 2024-09-16T13:01:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4489999999999996e-06 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039242+0000", 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7117000000000003e-05 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.039275+0000", 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.269e-05 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.041191+0000", 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019155470000000001 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.041233+0000", 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2571999999999998e-05 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:602 4.18 e148/147 4:18b02557:::rbd_data.11b54010928d.00000000000000aa:1b v 148'417, mlcod=148'413)", 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.049605+0000", 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3054674429999999, 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025643799999999998, 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049605+0000", 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049605+0000", 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049606+0000", 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2799999999999995e-07 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049612+0000", 2024-09-16T13:01:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.409e-06 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049630+0000", 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7592999999999999e-05 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.049635+0000", 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6870000000000002e-06 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050022+0000", 2024-09-16T13:01:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038735799999999997 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050040+0000", 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8542999999999998e-05 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.052129+0000", 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020881509999999999 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.052170+0000", 2024-09-16T13:01:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0911000000000002e-05 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:603 4.18 e148/147 4:1a8b08a8:::rbd_data.11b54010928d.0000000000000138:1b v 148'419, mlcod=148'413)", 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.050495+0000", 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.304577552, 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023761250000000002, 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050495+0000", 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050495+0000", 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050495+0000", 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6699999999999999e-07 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050499+0000", 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.709e-06 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050505+0000", 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7660000000000004e-06 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.050508+0000", 2024-09-16T13:01:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1209999999999998e-06 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051351+0000", 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084336899999999995 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051371+0000", 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9389e-05 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.052832+0000", 2024-09-16T13:01:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014615710000000001 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.052871+0000", 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8832999999999999e-05 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(osd.6.0:11 4.1f e148/147 ondisk, result = 0)", 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.051877+0000", 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.303196019, 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026759090000000002, 2024-09-16T13:01:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051877+0000", 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051877+0000", 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051877+0000", 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1500000000000004e-07 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051882+0000", 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3939999999999998e-06 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051893+0000", 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0906e-05 2024-09-16T13:01:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.051898+0000", 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1989999999999997e-06 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.053782+0000", 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001884265 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.054490+0000", 2024-09-16T13:01:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070751400000000002 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.054552+0000", 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2916000000000005e-05 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:605 4.18 e148/147 4:1c9f5a62:::rbd_data.11b515823229.00000000000000ff:1e v 148'423, mlcod=148'419)", 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.059635+0000", 2024-09-16T13:01:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.295437411, 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030075190000000002, 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059635+0000", 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059635+0000", 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059636+0000", 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.82e-07 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059641+0000", 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6450000000000002e-06 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059651+0000", 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6609999999999995e-06 2024-09-16T13:01:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059656+0000", 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059728+0000", 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1700999999999997e-05 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.059756+0000", 2024-09-16T13:01:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8459999999999999e-05 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.062610+0000", 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028539809999999998 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.062643+0000", 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2407999999999999e-05 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:606 4.18 e148/147 4:19e0f116:::rbd_data.11b515823229.00000000000000c7:1e v 148'425, mlcod=148'419)", 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.060608+0000", 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.294464499, 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002967969, 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060608+0000", 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060608+0000", 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060609+0000", 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5199999999999997e-07 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060615+0000", 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2024-09-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060636+0000", 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0913000000000001e-05 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060648+0000", 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1732e-05 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060796+0000", 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014830099999999999 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.060814+0000", 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8168999999999998e-05 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.063528+0000", 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027139989999999999 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.063576+0000", 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8016999999999998e-05 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4533.0:285148 4.a 4:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e148)", 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.069603+0000", 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.285469924, 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035980309999999998, 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4533", 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.42:0/1651980363", 2024-09-16T13:01:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 285148 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069603+0000", 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069603+0000", 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069605+0000", 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.968e-06 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069611+0000", 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.229e-06 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069623+0000", 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1779e-05 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069630+0000", 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9820000000000002e-06 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069754+0000", 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012427299999999999 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.069942+0000", 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000188097 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.072929+0000", 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029868579999999998 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.073032+0000", 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010353699999999999 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.073201+0000", 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016830799999999999 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(4.a 148/147 [PushOp(4:535385f0:::rbd_children:head, version: 146'1349, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 1, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:535385f0:::rbd_children:head@146'1349, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.070394+0000", 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.284678548, 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020175060000000001, 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070394+0000", 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070394+0000", 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070395+0000", 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3799999999999996e-07 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070413+0000", 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7934000000000001e-05 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070429+0000", 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6325999999999999e-05 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070439+0000", 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5279999999999992e-06 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070527+0000", 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8209999999999997e-05 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.070535+0000", 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5990000000000002e-06 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.072412+0000", 2024-09-16T13:01:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001876171 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:611 4.13 e148/147 4:caad81d8:::rbd_data.11b5b635ee69.00000000000001c5:17 v 148'490, mlcod=148'486)", 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.078727+0000", 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2763454850000002, 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022223099999999999, 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078727+0000", 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078727+0000", 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078728+0000", 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3200000000000005e-07 2024-09-16T13:01:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078735+0000", 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9820000000000002e-06 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078743+0000", 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1750000000000005e-06 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078748+0000", 2024-09-16T13:01:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4530000000000001e-06 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078889+0000", 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014086799999999999 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.078908+0000", 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8797999999999999e-05 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.080921+0000", 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002013239 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.080949+0000", 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8163000000000001e-05 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.425 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:612 4.13 e148/147 4:cfc5f29c:::rbd_data.11b5b635ee69.00000000000001aa:17 v 148'492, mlcod=148'486)", 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.079400+0000", 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2756721529999999, 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021213320000000001, 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079400+0000", 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079400+0000", 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079401+0000", 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6700000000000003e-07 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079407+0000", 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9920000000000004e-06 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079416+0000", 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9609999999999994e-06 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079421+0000", 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2050000000000001e-06 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079600+0000", 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017862999999999999 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.079619+0000", 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8817e-05 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.081487+0000", 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001868179 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.081522+0000", 2024-09-16T13:01:38.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4881e-05 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:613 4.13 e148/147 4:c99eeb3d:::rbd_data.11b5b3dc7343.0000000000000001:19 v 148'494, mlcod=148'490)", 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.084737+0000", 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2703357350000002, 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00215766, 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084737+0000", 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084737+0000", 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084738+0000", 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8700000000000005e-07 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084744+0000", 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.917e-06 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084787+0000", 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2045000000000001e-05 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084792+0000", 2024-09-16T13:01:38.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9549999999999997e-06 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084847+0000", 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.084868+0000", 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0616999999999999e-05 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.086859+0000", 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019913359999999998 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.086895+0000", 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5581999999999999e-05 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:617 4.13 e148/147 4:cba8a5ab:::rbd_data.11b5403f243a.0000000000000036:1f v 148'500, mlcod=148'494)", 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.096844+0000", 2024-09-16T13:01:38.434 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2582281339999999, 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026079720000000001, 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096844+0000", 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096844+0000", 2024-09-16T13:01:38.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096851+0000", 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0569999999999998e-06 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096855+0000", 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6970000000000001e-06 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096862+0000", 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0339999999999999e-06 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096873+0000", 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0902e-05 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096911+0000", 2024-09-16T13:01:38.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8331999999999998e-05 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.096930+0000", 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8403e-05 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.099411+0000", 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024809340000000002 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.099452+0000", 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1613000000000003e-05 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:625 4.e e148/147 4:76368bb3:::rbd_data.11b54010928d.0000000000000005:1b v 148'837, mlcod=148'833)", 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.128477+0000", 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2265951560000001, 2024-09-16T13:01:38.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021061410000000002, 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128477+0000", 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128477+0000", 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128478+0000", 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1399999999999996e-07 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128485+0000", 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5339999999999996e-06 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128497+0000", 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2311e-05 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128503+0000", 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9279999999999998e-06 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128555+0000", 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1858999999999999e-05 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.128585+0000", 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0125e-05 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.130534+0000", 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001948731 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.130584+0000", 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9839000000000002e-05 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:629 4.e e148/147 4:76f7a092:::rbd_data.11b5403f243a.0000000000000102:1f v 148'845, mlcod=148'841)", 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.139198+0000", 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2158748369999999, 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003046506, 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139198+0000", 2024-09-16T13:01:38.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139198+0000", 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139199+0000", 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9299999999999996e-07 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139205+0000", 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5050000000000004e-06 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139219+0000", 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3356999999999999e-05 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139224+0000", 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8640000000000001e-06 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139260+0000", 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5886999999999997e-05 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.139280+0000", 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9891999999999999e-05 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.142198+0000", 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029173749999999998 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.142244+0000", 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6733000000000001e-05 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.4.0:630 4.e e148/147 4:750dfe80:::rbd_data.11b5403f243a.0000000000000107:1f v 148'847, mlcod=148'841)", 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-16T13:01:35.140096+0000", 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.214977083, 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024568929999999999, 2024-09-16T13:01:38.448 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140096+0000", 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140096+0000", 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140096+0000", 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8899999999999998e-07 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140102+0000", 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6960000000000002e-06 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140114+0000", 2024-09-16T13:01:38.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1973e-05 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140120+0000", 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.57e-06 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140159+0000", 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8958999999999997e-05 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.140177+0000", 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8811e-05 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.142521+0000", 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00234323 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-16T13:01:35.142552+0000", 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1764999999999998e-05 2024-09-16T13:01:38.452 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.453 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-16T13:01:38.453 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-16T13:01:38.453 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-16T13:01:39.034 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T13:01:39.036 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-16T13:01:39.047 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T13:01:40.040 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T13:01:40.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T13:01:41.050 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T13:01:41.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T13:01:41.063 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T13:01:41.063 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:01:41.370 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:01:41.373 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:01:41.385 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-16T13:01:41.037755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590590,"num_objects":1006,"num_object_clones":0,"num_object_copies":2012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1001,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21119,"num_write_kb":309526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19274,"ondisk_log_size":19274,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422364,"kb_used_data":14284,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752552356,"statfs":{"total":773094113280,"available":770613612544,"internally_reserved":0,"allocated":14626816,"data_stored":8504752,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-137869858,"num_objects":-253,"num_object_clones":-103,"num_object_copies":-506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-254,"num_whiteouts":-83,"num_read":901,"num_read_kb":777,"num_write":195,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":-4,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.029197"},"pg_stats":[{"pgid":"4.1f","version":"150'338","reported_seq":4201,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.195460+0000","last_change":"2024-09-16T13:01:37.024157+0000","last_active":"2024-09-16T13:01:37.195460+0000","last_peered":"2024-09-16T13:01:37.195460+0000","last_clean":"2024-09-16T13:01:37.195460+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:01:37.195460+0000","last_undegraded":"2024-09-16T13:01:37.195460+0000","last_fullsized":"2024-09-16T13:01:37.195460+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:11.756487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002246297,"stat_sum":{"num_bytes":16,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":332,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"4.1e","version":"150'514","reported_seq":17213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027274+0000","last_change":"2024-09-16T13:01:38.062519+0000","last_active":"2024-09-16T13:01:39.027274+0000","last_peered":"2024-09-16T13:01:39.027274+0000","last_clean":"2024-09-16T13:01:39.027274+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:01:39.027274+0000","last_undegraded":"2024-09-16T13:01:39.027274+0000","last_fullsized":"2024-09-16T13:01:39.027274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1d","version":"142'433","reported_seq":4964,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.534114+0000","last_change":"2024-09-16T13:01:38.020672+0000","last_active":"2024-09-16T13:01:39.534114+0000","last_peered":"2024-09-16T13:01:39.534114+0000","last_clean":"2024-09-16T13:01:39.534114+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:01:39.534114+0000","last_undegraded":"2024-09-16T13:01:39.534114+0000","last_fullsized":"2024-09-16T13:01:39.534114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1c","version":"151'1171","reported_seq":7991,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755009+0000","last_change":"2024-09-16T13:01:38.017984+0000","last_active":"2024-09-16T13:01:39.755009+0000","last_peered":"2024-09-16T13:01:39.755009+0000","last_clean":"2024-09-16T13:01:39.755009+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:01:39.755009+0000","last_undegraded":"2024-09-16T13:01:39.755009+0000","last_fullsized":"2024-09-16T13:01:39.755009+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1b","version":"150'493","reported_seq":22019,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.026576+0000","last_change":"2024-09-16T13:01:38.026576+0000","last_active":"2024-09-16T13:01:38.026576+0000","last_peered":"2024-09-16T13:01:38.026576+0000","last_clean":"2024-09-16T13:01:38.026576+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:01:38.026576+0000","last_undegraded":"2024-09-16T13:01:38.026576+0000","last_fullsized":"2024-09-16T13:01:38.026576+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:04.244300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1a","version":"150'685","reported_seq":7122,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.127606+0000","last_change":"2024-09-16T13:01:38.026337+0000","last_active":"2024-09-16T13:01:38.127606+0000","last_peered":"2024-09-16T13:01:38.127606+0000","last_clean":"2024-09-16T13:01:38.127606+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:01:38.127606+0000","last_undegraded":"2024-09-16T13:01:38.127606+0000","last_fullsized":"2024-09-16T13:01:38.127606+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:30.964357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.19","version":"149'399","reported_seq":5454,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533841+0000","last_change":"2024-09-16T13:01:38.021608+0000","last_active":"2024-09-16T13:01:39.533841+0000","last_peered":"2024-09-16T13:01:39.533841+0000","last_clean":"2024-09-16T13:01:39.533841+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:01:39.533841+0000","last_undegraded":"2024-09-16T13:01:39.533841+0000","last_fullsized":"2024-09-16T13:01:39.533841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.18","version":"150'440","reported_seq":6436,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.024730+0000","last_change":"2024-09-16T13:01:38.024730+0000","last_active":"2024-09-16T13:01:38.024730+0000","last_peered":"2024-09-16T13:01:38.024730+0000","last_clean":"2024-09-16T13:01:38.024730+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:01:38.024730+0000","last_undegraded":"2024-09-16T13:01:38.024730+0000","last_fullsized":"2024-09-16T13:01:38.024730+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:23.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.17","version":"150'889","reported_seq":7611,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.534295+0000","last_change":"2024-09-16T13:01:38.021503+0000","last_active":"2024-09-16T13:01:39.534295+0000","last_peered":"2024-09-16T13:01:39.534295+0000","last_clean":"2024-09-16T13:01:39.534295+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:01:39.534295+0000","last_undegraded":"2024-09-16T13:01:39.534295+0000","last_fullsized":"2024-09-16T13:01:39.534295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.16","version":"150'508","reported_seq":3289,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027463+0000","last_change":"2024-09-16T13:01:38.062149+0000","last_active":"2024-09-16T13:01:39.027463+0000","last_peered":"2024-09-16T13:01:39.027463+0000","last_clean":"2024-09-16T13:01:39.027463+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:01:39.027463+0000","last_undegraded":"2024-09-16T13:01:39.027463+0000","last_fullsized":"2024-09-16T13:01:39.027463+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.15","version":"150'523","reported_seq":16193,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755469+0000","last_change":"2024-09-16T13:01:38.021224+0000","last_active":"2024-09-16T13:01:39.755469+0000","last_peered":"2024-09-16T13:01:39.755469+0000","last_clean":"2024-09-16T13:01:39.755469+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:01:39.755469+0000","last_undegraded":"2024-09-16T13:01:39.755469+0000","last_fullsized":"2024-09-16T13:01:39.755469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.14","version":"149'579","reported_seq":23207,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.534166+0000","last_change":"2024-09-16T13:01:38.020686+0000","last_active":"2024-09-16T13:01:39.534166+0000","last_peered":"2024-09-16T13:01:39.534166+0000","last_clean":"2024-09-16T13:01:39.534166+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:01:39.534166+0000","last_undegraded":"2024-09-16T13:01:39.534166+0000","last_fullsized":"2024-09-16T13:01:39.534166+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.13","version":"151'512","reported_seq":6212,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.150928+0000","last_change":"2024-09-16T13:01:38.027017+0000","last_active":"2024-09-16T13:01:38.150928+0000","last_peered":"2024-09-16T13:01:38.150928+0000","last_clean":"2024-09-16T13:01:38.150928+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:01:38.150928+0000","last_undegraded":"2024-09-16T13:01:38.150928+0000","last_fullsized":"2024-09-16T13:01:38.150928+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:56.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.12","version":"150'371","reported_seq":3102,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.754993+0000","last_change":"2024-09-16T13:01:38.019785+0000","last_active":"2024-09-16T13:01:39.754993+0000","last_peered":"2024-09-16T13:01:39.754993+0000","last_clean":"2024-09-16T13:01:39.754993+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:01:39.754993+0000","last_undegraded":"2024-09-16T13:01:39.754993+0000","last_fullsized":"2024-09-16T13:01:39.754993+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.11","version":"149'363","reported_seq":18807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533789+0000","last_change":"2024-09-16T13:01:38.019836+0000","last_active":"2024-09-16T13:01:39.533789+0000","last_peered":"2024-09-16T13:01:39.533789+0000","last_clean":"2024-09-16T13:01:39.533789+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:01:39.533789+0000","last_undegraded":"2024-09-16T13:01:39.533789+0000","last_fullsized":"2024-09-16T13:01:39.533789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.10","version":"139'335","reported_seq":5438,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533789+0000","last_change":"2024-09-16T13:01:38.018514+0000","last_active":"2024-09-16T13:01:39.533789+0000","last_peered":"2024-09-16T13:01:39.533789+0000","last_clean":"2024-09-16T13:01:39.533789+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:01:39.533789+0000","last_undegraded":"2024-09-16T13:01:39.533789+0000","last_fullsized":"2024-09-16T13:01:39.533789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T13:01:36.009413+0000","last_change":"2024-09-16T13:01:35.025514+0000","last_active":"2024-09-16T13:01:36.009413+0000","last_peered":"2024-09-16T13:01:36.009413+0000","last_clean":"2024-09-16T13:01:36.009413+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:01:36.009413+0000","last_undegraded":"2024-09-16T13:01:36.009413+0000","last_fullsized":"2024-09-16T13:01:36.009413+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:27.902219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"146'451","reported_seq":3755,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.027021+0000","last_change":"2024-09-16T13:01:38.027021+0000","last_active":"2024-09-16T13:01:38.027021+0000","last_peered":"2024-09-16T13:01:38.027021+0000","last_clean":"2024-09-16T13:01:38.027021+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:01:38.027021+0000","last_undegraded":"2024-09-16T13:01:38.027021+0000","last_fullsized":"2024-09-16T13:01:38.027021+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.6","version":"0'0","reported_seq":117,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.114484+0000","last_change":"2024-09-16T13:01:34.980687+0000","last_active":"2024-09-16T13:01:37.114484+0000","last_peered":"2024-09-16T13:01:37.114484+0000","last_clean":"2024-09-16T13:01:37.114484+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:01:37.114484+0000","last_undegraded":"2024-09-16T13:01:37.114484+0000","last_fullsized":"2024-09-16T13:01:37.114484+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:44.616572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"149'690","reported_seq":7307,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533791+0000","last_change":"2024-09-16T13:01:38.020063+0000","last_active":"2024-09-16T13:01:39.533791+0000","last_peered":"2024-09-16T13:01:39.533791+0000","last_clean":"2024-09-16T13:01:39.533791+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:01:39.533791+0000","last_undegraded":"2024-09-16T13:01:39.533791+0000","last_fullsized":"2024-09-16T13:01:39.533791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.5","version":"0'0","reported_seq":144,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.166655+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:01:38.166655+0000","last_peered":"2024-09-16T13:01:38.166655+0000","last_clean":"2024-09-16T13:01:38.166655+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:01:38.166655+0000","last_undegraded":"2024-09-16T13:01:38.166655+0000","last_fullsized":"2024-09-16T13:01:38.166655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"151'687","reported_seq":3198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027087+0000","last_change":"2024-09-16T13:01:38.061542+0000","last_active":"2024-09-16T13:01:39.027087+0000","last_peered":"2024-09-16T13:01:39.027087+0000","last_clean":"2024-09-16T13:01:39.027087+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:01:39.027087+0000","last_undegraded":"2024-09-16T13:01:39.027087+0000","last_fullsized":"2024-09-16T13:01:39.027087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.4","version":"0'0","reported_seq":102,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.114188+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:01:37.114188+0000","last_peered":"2024-09-16T13:01:37.114188+0000","last_clean":"2024-09-16T13:01:37.114188+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:01:37.114188+0000","last_undegraded":"2024-09-16T13:01:37.114188+0000","last_fullsized":"2024-09-16T13:01:37.114188+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"150'586","reported_seq":17088,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.113796+0000","last_change":"2024-09-16T13:01:38.114806+0000","last_active":"2024-09-16T13:01:38.113796+0000","last_peered":"2024-09-16T13:01:38.113796+0000","last_clean":"2024-09-16T13:01:38.113796+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:01:38.113796+0000","last_undegraded":"2024-09-16T13:01:38.113796+0000","last_fullsized":"2024-09-16T13:01:38.113796+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"3.3","version":"0'0","reported_seq":106,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.023883+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:01:38.023883+0000","last_peered":"2024-09-16T13:01:38.023883+0000","last_clean":"2024-09-16T13:01:38.023883+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:01:38.023883+0000","last_undegraded":"2024-09-16T13:01:38.023883+0000","last_fullsized":"2024-09-16T13:01:38.023883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":172,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755578+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:01:39.755578+0000","last_peered":"2024-09-16T13:01:39.755578+0000","last_clean":"2024-09-16T13:01:39.755578+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:01:39.755578+0000","last_undegraded":"2024-09-16T13:01:39.755578+0000","last_fullsized":"2024-09-16T13:01:39.755578+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"148'501","reported_seq":17910,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.021958+0000","last_change":"2024-09-16T13:01:37.022816+0000","last_active":"2024-09-16T13:01:37.021958+0000","last_peered":"2024-09-16T13:01:37.021958+0000","last_clean":"2024-09-16T13:01:37.021958+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:01:37.021958+0000","last_undegraded":"2024-09-16T13:01:37.021958+0000","last_fullsized":"2024-09-16T13:01:37.021958+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:27.630481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010551410000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"3.0","version":"0'0","reported_seq":102,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.113765+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:01:37.113765+0000","last_peered":"2024-09-16T13:01:37.113765+0000","last_clean":"2024-09-16T13:01:37.113765+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:01:37.113765+0000","last_undegraded":"2024-09-16T13:01:37.113765+0000","last_fullsized":"2024-09-16T13:01:37.113765+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":147,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.754784+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:01:39.754784+0000","last_peered":"2024-09-16T13:01:39.754784+0000","last_clean":"2024-09-16T13:01:39.754784+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:01:39.754784+0000","last_undegraded":"2024-09-16T13:01:39.754784+0000","last_fullsized":"2024-09-16T13:01:39.754784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'407","reported_seq":2753,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.113285+0000","last_change":"2024-09-16T13:01:38.113780+0000","last_active":"2024-09-16T13:01:38.113285+0000","last_peered":"2024-09-16T13:01:38.113285+0000","last_clean":"2024-09-16T13:01:38.113285+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:01:38.113285+0000","last_undegraded":"2024-09-16T13:01:38.113285+0000","last_fullsized":"2024-09-16T13:01:38.113285+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"3.1","version":"55'1","reported_seq":150,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.026965+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:01:39.026965+0000","last_peered":"2024-09-16T13:01:39.026965+0000","last_clean":"2024-09-16T13:01:39.026965+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:01:39.026965+0000","last_undegraded":"2024-09-16T13:01:39.026965+0000","last_fullsized":"2024-09-16T13:01:39.026965+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.166779+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:01:38.166779+0000","last_peered":"2024-09-16T13:01:38.166779+0000","last_clean":"2024-09-16T13:01:38.166779+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:01:38.166779+0000","last_undegraded":"2024-09-16T13:01:38.166779+0000","last_fullsized":"2024-09-16T13:01:38.166779+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"151'434","reported_seq":7301,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027656+0000","last_change":"2024-09-16T13:01:38.062717+0000","last_active":"2024-09-16T13:01:39.027656+0000","last_peered":"2024-09-16T13:01:39.027656+0000","last_clean":"2024-09-16T13:01:39.027656+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:01:39.027656+0000","last_undegraded":"2024-09-16T13:01:39.027656+0000","last_fullsized":"2024-09-16T13:01:39.027656+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"3.2","version":"151'4","reported_seq":140,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533958+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:01:39.533958+0000","last_peered":"2024-09-16T13:01:39.533958+0000","last_clean":"2024-09-16T13:01:39.533958+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:01:39.533958+0000","last_undegraded":"2024-09-16T13:01:39.533958+0000","last_fullsized":"2024-09-16T13:01:39.533958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":145,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755376+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:01:39.755376+0000","last_peered":"2024-09-16T13:01:39.755376+0000","last_clean":"2024-09-16T13:01:39.755376+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:01:39.755376+0000","last_undegraded":"2024-09-16T13:01:39.755376+0000","last_fullsized":"2024-09-16T13:01:39.755376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":232,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.166950+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:01:38.166950+0000","last_peered":"2024-09-16T13:01:38.166950+0000","last_clean":"2024-09-16T13:01:38.166950+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:01:38.166950+0000","last_undegraded":"2024-09-16T13:01:38.166950+0000","last_fullsized":"2024-09-16T13:01:38.166950+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"150'444","reported_seq":3693,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.022639+0000","last_change":"2024-09-16T13:01:37.022639+0000","last_active":"2024-09-16T13:01:37.022639+0000","last_peered":"2024-09-16T13:01:37.022639+0000","last_clean":"2024-09-16T13:01:37.022639+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:01:37.022639+0000","last_undegraded":"2024-09-16T13:01:37.022639+0000","last_fullsized":"2024-09-16T13:01:37.022639+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:16.451880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090728,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"4.8","version":"150'977","reported_seq":18761,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.754914+0000","last_change":"2024-09-16T13:01:38.019797+0000","last_active":"2024-09-16T13:01:39.754914+0000","last_peered":"2024-09-16T13:01:39.754914+0000","last_clean":"2024-09-16T13:01:39.754914+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:01:39.754914+0000","last_undegraded":"2024-09-16T13:01:39.754914+0000","last_fullsized":"2024-09-16T13:01:39.754914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.9","version":"149'328","reported_seq":827,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.024649+0000","last_change":"2024-09-16T13:01:38.024649+0000","last_active":"2024-09-16T13:01:38.024649+0000","last_peered":"2024-09-16T13:01:38.024649+0000","last_clean":"2024-09-16T13:01:38.024649+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:01:38.024649+0000","last_undegraded":"2024-09-16T13:01:38.024649+0000","last_fullsized":"2024-09-16T13:01:38.024649+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.a","version":"150'1380","reported_seq":6945,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.023343+0000","last_change":"2024-09-16T13:01:37.023343+0000","last_active":"2024-09-16T13:01:37.023343+0000","last_peered":"2024-09-16T13:01:37.023343+0000","last_clean":"2024-09-16T13:01:37.023343+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:01:37.023343+0000","last_undegraded":"2024-09-16T13:01:37.023343+0000","last_fullsized":"2024-09-16T13:01:37.023343+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:00.052447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014800519999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"4.b","version":"144'974","reported_seq":6616,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027095+0000","last_change":"2024-09-16T13:01:38.060958+0000","last_active":"2024-09-16T13:01:39.027095+0000","last_peered":"2024-09-16T13:01:39.027095+0000","last_clean":"2024-09-16T13:01:39.027095+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:01:39.027095+0000","last_undegraded":"2024-09-16T13:01:39.027095+0000","last_fullsized":"2024-09-16T13:01:39.027095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.c","version":"151'553","reported_seq":5330,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.148180+0000","last_change":"2024-09-16T13:01:38.025883+0000","last_active":"2024-09-16T13:01:38.148180+0000","last_peered":"2024-09-16T13:01:38.148180+0000","last_clean":"2024-09-16T13:01:38.148180+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:01:38.148180+0000","last_undegraded":"2024-09-16T13:01:38.148180+0000","last_fullsized":"2024-09-16T13:01:38.148180+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.d","version":"150'434","reported_seq":4385,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.027791+0000","last_change":"2024-09-16T13:01:38.027791+0000","last_active":"2024-09-16T13:01:38.027791+0000","last_peered":"2024-09-16T13:01:38.027791+0000","last_clean":"2024-09-16T13:01:38.027791+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:01:38.027791+0000","last_undegraded":"2024-09-16T13:01:38.027791+0000","last_fullsized":"2024-09-16T13:01:38.027791+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.e","version":"150'876","reported_seq":4031,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.025990+0000","last_change":"2024-09-16T13:01:38.025990+0000","last_active":"2024-09-16T13:01:38.025990+0000","last_peered":"2024-09-16T13:01:38.025990+0000","last_clean":"2024-09-16T13:01:38.025990+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:01:38.025990+0000","last_undegraded":"2024-09-16T13:01:38.025990+0000","last_fullsized":"2024-09-16T13:01:38.025990+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:12.558678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.f","version":"149'919","reported_seq":36727,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.113134+0000","last_change":"2024-09-16T13:01:38.113695+0000","last_active":"2024-09-16T13:01:38.113134+0000","last_peered":"2024-09-16T13:01:38.113134+0000","last_clean":"2024-09-16T13:01:38.113134+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:01:38.113134+0000","last_undegraded":"2024-09-16T13:01:38.113134+0000","last_fullsized":"2024-09-16T13:01:38.113134+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1d"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":35,"num_objects":997,"num_object_clones":0,"num_object_copies":1994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":997,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21014,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":89,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":19194,"ondisk_log_size":19194,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192514,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301560,"kb_used_data":1760,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070280,"statfs":{"total":96636764160,"available":96327966720,"internally_reserved":0,"allocated":1802240,"data_stored":907367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303328,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068512,"statfs":{"total":96636764160,"available":96326156288,"internally_reserved":0,"allocated":1818624,"data_stored":914173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.224,"5min":1.272,"15min":1.272},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2130000000000001,"5min":1.391,"15min":1.391},"last":1.012}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":0.96899999999999997,"15min":0.89100000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.275,"5min":14.275,"15min":14.275},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.91100000000000003}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.999,"5min":1.095,"15min":1.111},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.097,"15min":1.075},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.0429999999999999}]},{"osd":4,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0720000000000001}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":1.137}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.1160000000000001,"15min":1.1499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4399999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.024,"5min":1.224,"15min":1.258},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.53,"5min":1.748,"15min":1.748},"last":1.2190000000000001}]}]},{"osd":4,"up_from":108,"seq":463856467978,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309192,"kb_used_data":1768,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062648,"statfs":{"total":96636764160,"available":96320151552,"internally_reserved":0,"allocated":1810432,"data_stored":914005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299144,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072696,"statfs":{"total":96636764160,"available":96330440704,"internally_reserved":0,"allocated":1830912,"data_stored":920449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.33900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.79400000000000004}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":1.091},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76900000000000002,"15min":0.77800000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.65400000000000003}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.82599999999999996,"15min":0.80500000000000005},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.752}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.278}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1319999999999999,"15min":1.1359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.097,"15min":1.115},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.3700000000000001}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":0.995},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.2370000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.113,"15min":1.1279999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.208,"15min":1.1919999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.1950000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299116,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072724,"statfs":{"total":96636764160,"available":96330469376,"internally_reserved":0,"allocated":1802240,"data_stored":913493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.80000000000000004,"15min":0.75900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.654,"5min":10.654,"15min":10.654},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.244,"5min":0.84599999999999997,"15min":0.78000000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.28799999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.88400000000000001,"15min":0.86199999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":0.90700000000000003,"15min":0.80900000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":18.117000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.88300000000000001,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.56999999999999995}]},{"osd":4,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.44800000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.159,"15min":1.2150000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.415,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.878,"5min":1.1180000000000001,"15min":1.1579999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5509999999999999,"5min":2.125,"15min":2.125},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.621}]},{"osd":7,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.2010000000000001,"15min":1.2490000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.792,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.2010000000000001,"15min":1.2609999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3899999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.63700000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":1888256,"data_stored":1516906,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":0.91300000000000003,"15min":0.85099999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.407,"5min":9.407,"15min":9.407},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.91100000000000003,"15min":0.96299999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.99399999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.123,"5min":2.254,"15min":2.254},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.80100000000000005,"15min":0.79300000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.3460000000000001,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.75},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.997,"15min":1.0429999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.256,"5min":2.044,"15min":2.044},"last":0.85599999999999998}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.0449999999999999,"15min":1.0740000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4370000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.1819999999999999,"15min":1.222},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.617,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.60999999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.96499999999999997}]},{"osd":7,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.163,"15min":1.1879999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6779999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.1319999999999999,"15min":1.167},"min":{"1min":0.438,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.583,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.878}]}]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303372,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068468,"statfs":{"total":96636764160,"available":96326111232,"internally_reserved":0,"allocated":1863680,"data_stored":1504354,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.03},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.2370000000000001,"15min":1.2909999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7470000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.125,"15min":1.1890000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.681,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.505},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.115,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.94899999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.071,"15min":1.069},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.2110000000000001,"15min":1.2250000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.147}]},{"osd":3,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0449999999999999,"15min":1.083},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2869999999999999,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.984},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.2649999999999999,"15min":1.319},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.623,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.9299999999999999}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.7889999999999997,"5min":4.7889999999999997,"15min":4.7889999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.82599999999999996}]},{"osd":5,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.284,"15min":1.339},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.75,"5min":6.282,"15min":6.282},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.96799999999999997,"15min":0.98799999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.611,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.89800000000000002}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.64400000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1810432,"data_stored":914005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85099999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.51500000000000001}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76600000000000001,"15min":0.78900000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.234,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.86199999999999999,"15min":0.88800000000000001},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.077,"5min":3.024,"15min":3.024},"last":0.68300000000000005}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":1.004,"15min":1.048},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.103,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.76800000000000002}]},{"osd":4,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.349,"15min":1.431},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.234,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.071,"15min":1.0880000000000001},"min":{"1min":0.59799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.79200000000000004}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.077}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1299999999999999,"15min":1.147},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.997},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.27,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.1140000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":51,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:01:41.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:01:41.701 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:01:41.704 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:01:41.717 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-16T13:01:41.037755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590590,"num_objects":1006,"num_object_clones":0,"num_object_copies":2012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1001,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21119,"num_write_kb":309526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19274,"ondisk_log_size":19274,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422364,"kb_used_data":14284,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752552356,"statfs":{"total":773094113280,"available":770613612544,"internally_reserved":0,"allocated":14626816,"data_stored":8504752,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-137869858,"num_objects":-253,"num_object_clones":-103,"num_object_copies":-506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-254,"num_whiteouts":-83,"num_read":901,"num_read_kb":777,"num_write":195,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":-4,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.029197"},"pg_stats":[{"pgid":"4.1f","version":"150'338","reported_seq":4201,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.195460+0000","last_change":"2024-09-16T13:01:37.024157+0000","last_active":"2024-09-16T13:01:37.195460+0000","last_peered":"2024-09-16T13:01:37.195460+0000","last_clean":"2024-09-16T13:01:37.195460+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:01:37.195460+0000","last_undegraded":"2024-09-16T13:01:37.195460+0000","last_fullsized":"2024-09-16T13:01:37.195460+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:11.756487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002246297,"stat_sum":{"num_bytes":16,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":332,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"4.1e","version":"150'514","reported_seq":17213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027274+0000","last_change":"2024-09-16T13:01:38.062519+0000","last_active":"2024-09-16T13:01:39.027274+0000","last_peered":"2024-09-16T13:01:39.027274+0000","last_clean":"2024-09-16T13:01:39.027274+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:01:39.027274+0000","last_undegraded":"2024-09-16T13:01:39.027274+0000","last_fullsized":"2024-09-16T13:01:39.027274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1d","version":"142'433","reported_seq":4964,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.534114+0000","last_change":"2024-09-16T13:01:38.020672+0000","last_active":"2024-09-16T13:01:39.534114+0000","last_peered":"2024-09-16T13:01:39.534114+0000","last_clean":"2024-09-16T13:01:39.534114+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:01:39.534114+0000","last_undegraded":"2024-09-16T13:01:39.534114+0000","last_fullsized":"2024-09-16T13:01:39.534114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1c","version":"151'1171","reported_seq":7991,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755009+0000","last_change":"2024-09-16T13:01:38.017984+0000","last_active":"2024-09-16T13:01:39.755009+0000","last_peered":"2024-09-16T13:01:39.755009+0000","last_clean":"2024-09-16T13:01:39.755009+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:01:39.755009+0000","last_undegraded":"2024-09-16T13:01:39.755009+0000","last_fullsized":"2024-09-16T13:01:39.755009+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1b","version":"150'493","reported_seq":22019,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.026576+0000","last_change":"2024-09-16T13:01:38.026576+0000","last_active":"2024-09-16T13:01:38.026576+0000","last_peered":"2024-09-16T13:01:38.026576+0000","last_clean":"2024-09-16T13:01:38.026576+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:01:38.026576+0000","last_undegraded":"2024-09-16T13:01:38.026576+0000","last_fullsized":"2024-09-16T13:01:38.026576+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:04.244300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1a","version":"150'685","reported_seq":7122,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.127606+0000","last_change":"2024-09-16T13:01:38.026337+0000","last_active":"2024-09-16T13:01:38.127606+0000","last_peered":"2024-09-16T13:01:38.127606+0000","last_clean":"2024-09-16T13:01:38.127606+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:01:38.127606+0000","last_undegraded":"2024-09-16T13:01:38.127606+0000","last_fullsized":"2024-09-16T13:01:38.127606+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:30.964357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.19","version":"149'399","reported_seq":5454,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533841+0000","last_change":"2024-09-16T13:01:38.021608+0000","last_active":"2024-09-16T13:01:39.533841+0000","last_peered":"2024-09-16T13:01:39.533841+0000","last_clean":"2024-09-16T13:01:39.533841+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:01:39.533841+0000","last_undegraded":"2024-09-16T13:01:39.533841+0000","last_fullsized":"2024-09-16T13:01:39.533841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.18","version":"150'440","reported_seq":6436,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.024730+0000","last_change":"2024-09-16T13:01:38.024730+0000","last_active":"2024-09-16T13:01:38.024730+0000","last_peered":"2024-09-16T13:01:38.024730+0000","last_clean":"2024-09-16T13:01:38.024730+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:01:38.024730+0000","last_undegraded":"2024-09-16T13:01:38.024730+0000","last_fullsized":"2024-09-16T13:01:38.024730+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:23.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.17","version":"150'889","reported_seq":7611,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.534295+0000","last_change":"2024-09-16T13:01:38.021503+0000","last_active":"2024-09-16T13:01:39.534295+0000","last_peered":"2024-09-16T13:01:39.534295+0000","last_clean":"2024-09-16T13:01:39.534295+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:01:39.534295+0000","last_undegraded":"2024-09-16T13:01:39.534295+0000","last_fullsized":"2024-09-16T13:01:39.534295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.16","version":"150'508","reported_seq":3289,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027463+0000","last_change":"2024-09-16T13:01:38.062149+0000","last_active":"2024-09-16T13:01:39.027463+0000","last_peered":"2024-09-16T13:01:39.027463+0000","last_clean":"2024-09-16T13:01:39.027463+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:01:39.027463+0000","last_undegraded":"2024-09-16T13:01:39.027463+0000","last_fullsized":"2024-09-16T13:01:39.027463+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.15","version":"150'523","reported_seq":16193,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755469+0000","last_change":"2024-09-16T13:01:38.021224+0000","last_active":"2024-09-16T13:01:39.755469+0000","last_peered":"2024-09-16T13:01:39.755469+0000","last_clean":"2024-09-16T13:01:39.755469+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:01:39.755469+0000","last_undegraded":"2024-09-16T13:01:39.755469+0000","last_fullsized":"2024-09-16T13:01:39.755469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.14","version":"149'579","reported_seq":23207,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.534166+0000","last_change":"2024-09-16T13:01:38.020686+0000","last_active":"2024-09-16T13:01:39.534166+0000","last_peered":"2024-09-16T13:01:39.534166+0000","last_clean":"2024-09-16T13:01:39.534166+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:01:39.534166+0000","last_undegraded":"2024-09-16T13:01:39.534166+0000","last_fullsized":"2024-09-16T13:01:39.534166+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.13","version":"151'512","reported_seq":6212,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.150928+0000","last_change":"2024-09-16T13:01:38.027017+0000","last_active":"2024-09-16T13:01:38.150928+0000","last_peered":"2024-09-16T13:01:38.150928+0000","last_clean":"2024-09-16T13:01:38.150928+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:01:38.150928+0000","last_undegraded":"2024-09-16T13:01:38.150928+0000","last_fullsized":"2024-09-16T13:01:38.150928+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:56.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.12","version":"150'371","reported_seq":3102,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.754993+0000","last_change":"2024-09-16T13:01:38.019785+0000","last_active":"2024-09-16T13:01:39.754993+0000","last_peered":"2024-09-16T13:01:39.754993+0000","last_clean":"2024-09-16T13:01:39.754993+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:01:39.754993+0000","last_undegraded":"2024-09-16T13:01:39.754993+0000","last_fullsized":"2024-09-16T13:01:39.754993+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.11","version":"149'363","reported_seq":18807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533789+0000","last_change":"2024-09-16T13:01:38.019836+0000","last_active":"2024-09-16T13:01:39.533789+0000","last_peered":"2024-09-16T13:01:39.533789+0000","last_clean":"2024-09-16T13:01:39.533789+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:01:39.533789+0000","last_undegraded":"2024-09-16T13:01:39.533789+0000","last_fullsized":"2024-09-16T13:01:39.533789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.10","version":"139'335","reported_seq":5438,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533789+0000","last_change":"2024-09-16T13:01:38.018514+0000","last_active":"2024-09-16T13:01:39.533789+0000","last_peered":"2024-09-16T13:01:39.533789+0000","last_clean":"2024-09-16T13:01:39.533789+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:01:39.533789+0000","last_undegraded":"2024-09-16T13:01:39.533789+0000","last_fullsized":"2024-09-16T13:01:39.533789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T13:01:36.009413+0000","last_change":"2024-09-16T13:01:35.025514+0000","last_active":"2024-09-16T13:01:36.009413+0000","last_peered":"2024-09-16T13:01:36.009413+0000","last_clean":"2024-09-16T13:01:36.009413+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:01:36.009413+0000","last_undegraded":"2024-09-16T13:01:36.009413+0000","last_fullsized":"2024-09-16T13:01:36.009413+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:27.902219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"146'451","reported_seq":3755,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.027021+0000","last_change":"2024-09-16T13:01:38.027021+0000","last_active":"2024-09-16T13:01:38.027021+0000","last_peered":"2024-09-16T13:01:38.027021+0000","last_clean":"2024-09-16T13:01:38.027021+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:01:38.027021+0000","last_undegraded":"2024-09-16T13:01:38.027021+0000","last_fullsized":"2024-09-16T13:01:38.027021+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.6","version":"0'0","reported_seq":117,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.114484+0000","last_change":"2024-09-16T13:01:34.980687+0000","last_active":"2024-09-16T13:01:37.114484+0000","last_peered":"2024-09-16T13:01:37.114484+0000","last_clean":"2024-09-16T13:01:37.114484+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:01:37.114484+0000","last_undegraded":"2024-09-16T13:01:37.114484+0000","last_fullsized":"2024-09-16T13:01:37.114484+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:44.616572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"149'690","reported_seq":7307,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533791+0000","last_change":"2024-09-16T13:01:38.020063+0000","last_active":"2024-09-16T13:01:39.533791+0000","last_peered":"2024-09-16T13:01:39.533791+0000","last_clean":"2024-09-16T13:01:39.533791+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:01:39.533791+0000","last_undegraded":"2024-09-16T13:01:39.533791+0000","last_fullsized":"2024-09-16T13:01:39.533791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.5","version":"0'0","reported_seq":144,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.166655+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:01:38.166655+0000","last_peered":"2024-09-16T13:01:38.166655+0000","last_clean":"2024-09-16T13:01:38.166655+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:01:38.166655+0000","last_undegraded":"2024-09-16T13:01:38.166655+0000","last_fullsized":"2024-09-16T13:01:38.166655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"151'687","reported_seq":3198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027087+0000","last_change":"2024-09-16T13:01:38.061542+0000","last_active":"2024-09-16T13:01:39.027087+0000","last_peered":"2024-09-16T13:01:39.027087+0000","last_clean":"2024-09-16T13:01:39.027087+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:01:39.027087+0000","last_undegraded":"2024-09-16T13:01:39.027087+0000","last_fullsized":"2024-09-16T13:01:39.027087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.4","version":"0'0","reported_seq":102,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.114188+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:01:37.114188+0000","last_peered":"2024-09-16T13:01:37.114188+0000","last_clean":"2024-09-16T13:01:37.114188+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:01:37.114188+0000","last_undegraded":"2024-09-16T13:01:37.114188+0000","last_fullsized":"2024-09-16T13:01:37.114188+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"150'586","reported_seq":17088,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.113796+0000","last_change":"2024-09-16T13:01:38.114806+0000","last_active":"2024-09-16T13:01:38.113796+0000","last_peered":"2024-09-16T13:01:38.113796+0000","last_clean":"2024-09-16T13:01:38.113796+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:01:38.113796+0000","last_undegraded":"2024-09-16T13:01:38.113796+0000","last_fullsized":"2024-09-16T13:01:38.113796+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"3.3","version":"0'0","reported_seq":106,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.023883+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:01:38.023883+0000","last_peered":"2024-09-16T13:01:38.023883+0000","last_clean":"2024-09-16T13:01:38.023883+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:01:38.023883+0000","last_undegraded":"2024-09-16T13:01:38.023883+0000","last_fullsized":"2024-09-16T13:01:38.023883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":172,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755578+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:01:39.755578+0000","last_peered":"2024-09-16T13:01:39.755578+0000","last_clean":"2024-09-16T13:01:39.755578+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:01:39.755578+0000","last_undegraded":"2024-09-16T13:01:39.755578+0000","last_fullsized":"2024-09-16T13:01:39.755578+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"148'501","reported_seq":17910,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.021958+0000","last_change":"2024-09-16T13:01:37.022816+0000","last_active":"2024-09-16T13:01:37.021958+0000","last_peered":"2024-09-16T13:01:37.021958+0000","last_clean":"2024-09-16T13:01:37.021958+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:01:37.021958+0000","last_undegraded":"2024-09-16T13:01:37.021958+0000","last_fullsized":"2024-09-16T13:01:37.021958+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:27.630481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010551410000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"3.0","version":"0'0","reported_seq":102,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.113765+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:01:37.113765+0000","last_peered":"2024-09-16T13:01:37.113765+0000","last_clean":"2024-09-16T13:01:37.113765+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:01:37.113765+0000","last_undegraded":"2024-09-16T13:01:37.113765+0000","last_fullsized":"2024-09-16T13:01:37.113765+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":147,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.754784+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:01:39.754784+0000","last_peered":"2024-09-16T13:01:39.754784+0000","last_clean":"2024-09-16T13:01:39.754784+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:01:39.754784+0000","last_undegraded":"2024-09-16T13:01:39.754784+0000","last_fullsized":"2024-09-16T13:01:39.754784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'407","reported_seq":2753,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.113285+0000","last_change":"2024-09-16T13:01:38.113780+0000","last_active":"2024-09-16T13:01:38.113285+0000","last_peered":"2024-09-16T13:01:38.113285+0000","last_clean":"2024-09-16T13:01:38.113285+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:01:38.113285+0000","last_undegraded":"2024-09-16T13:01:38.113285+0000","last_fullsized":"2024-09-16T13:01:38.113285+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"3.1","version":"55'1","reported_seq":150,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.026965+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:01:39.026965+0000","last_peered":"2024-09-16T13:01:39.026965+0000","last_clean":"2024-09-16T13:01:39.026965+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:01:39.026965+0000","last_undegraded":"2024-09-16T13:01:39.026965+0000","last_fullsized":"2024-09-16T13:01:39.026965+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.166779+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:01:38.166779+0000","last_peered":"2024-09-16T13:01:38.166779+0000","last_clean":"2024-09-16T13:01:38.166779+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:01:38.166779+0000","last_undegraded":"2024-09-16T13:01:38.166779+0000","last_fullsized":"2024-09-16T13:01:38.166779+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"151'434","reported_seq":7301,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027656+0000","last_change":"2024-09-16T13:01:38.062717+0000","last_active":"2024-09-16T13:01:39.027656+0000","last_peered":"2024-09-16T13:01:39.027656+0000","last_clean":"2024-09-16T13:01:39.027656+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:01:39.027656+0000","last_undegraded":"2024-09-16T13:01:39.027656+0000","last_fullsized":"2024-09-16T13:01:39.027656+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"3.2","version":"151'4","reported_seq":140,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533958+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:01:39.533958+0000","last_peered":"2024-09-16T13:01:39.533958+0000","last_clean":"2024-09-16T13:01:39.533958+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:01:39.533958+0000","last_undegraded":"2024-09-16T13:01:39.533958+0000","last_fullsized":"2024-09-16T13:01:39.533958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":145,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755376+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:01:39.755376+0000","last_peered":"2024-09-16T13:01:39.755376+0000","last_clean":"2024-09-16T13:01:39.755376+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:01:39.755376+0000","last_undegraded":"2024-09-16T13:01:39.755376+0000","last_fullsized":"2024-09-16T13:01:39.755376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":232,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.166950+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:01:38.166950+0000","last_peered":"2024-09-16T13:01:38.166950+0000","last_clean":"2024-09-16T13:01:38.166950+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:01:38.166950+0000","last_undegraded":"2024-09-16T13:01:38.166950+0000","last_fullsized":"2024-09-16T13:01:38.166950+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"150'444","reported_seq":3693,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.022639+0000","last_change":"2024-09-16T13:01:37.022639+0000","last_active":"2024-09-16T13:01:37.022639+0000","last_peered":"2024-09-16T13:01:37.022639+0000","last_clean":"2024-09-16T13:01:37.022639+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:01:37.022639+0000","last_undegraded":"2024-09-16T13:01:37.022639+0000","last_fullsized":"2024-09-16T13:01:37.022639+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:16.451880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090728,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"4.8","version":"150'977","reported_seq":18761,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.754914+0000","last_change":"2024-09-16T13:01:38.019797+0000","last_active":"2024-09-16T13:01:39.754914+0000","last_peered":"2024-09-16T13:01:39.754914+0000","last_clean":"2024-09-16T13:01:39.754914+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:01:39.754914+0000","last_undegraded":"2024-09-16T13:01:39.754914+0000","last_fullsized":"2024-09-16T13:01:39.754914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.9","version":"149'328","reported_seq":827,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.024649+0000","last_change":"2024-09-16T13:01:38.024649+0000","last_active":"2024-09-16T13:01:38.024649+0000","last_peered":"2024-09-16T13:01:38.024649+0000","last_clean":"2024-09-16T13:01:38.024649+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:01:38.024649+0000","last_undegraded":"2024-09-16T13:01:38.024649+0000","last_fullsized":"2024-09-16T13:01:38.024649+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.a","version":"150'1380","reported_seq":6945,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.023343+0000","last_change":"2024-09-16T13:01:37.023343+0000","last_active":"2024-09-16T13:01:37.023343+0000","last_peered":"2024-09-16T13:01:37.023343+0000","last_clean":"2024-09-16T13:01:37.023343+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:01:37.023343+0000","last_undegraded":"2024-09-16T13:01:37.023343+0000","last_fullsized":"2024-09-16T13:01:37.023343+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:00.052447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014800519999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"4.b","version":"144'974","reported_seq":6616,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027095+0000","last_change":"2024-09-16T13:01:38.060958+0000","last_active":"2024-09-16T13:01:39.027095+0000","last_peered":"2024-09-16T13:01:39.027095+0000","last_clean":"2024-09-16T13:01:39.027095+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:01:39.027095+0000","last_undegraded":"2024-09-16T13:01:39.027095+0000","last_fullsized":"2024-09-16T13:01:39.027095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.c","version":"151'553","reported_seq":5330,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.148180+0000","last_change":"2024-09-16T13:01:38.025883+0000","last_active":"2024-09-16T13:01:38.148180+0000","last_peered":"2024-09-16T13:01:38.148180+0000","last_clean":"2024-09-16T13:01:38.148180+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:01:38.148180+0000","last_undegraded":"2024-09-16T13:01:38.148180+0000","last_fullsized":"2024-09-16T13:01:38.148180+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.d","version":"150'434","reported_seq":4385,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.027791+0000","last_change":"2024-09-16T13:01:38.027791+0000","last_active":"2024-09-16T13:01:38.027791+0000","last_peered":"2024-09-16T13:01:38.027791+0000","last_clean":"2024-09-16T13:01:38.027791+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:01:38.027791+0000","last_undegraded":"2024-09-16T13:01:38.027791+0000","last_fullsized":"2024-09-16T13:01:38.027791+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.e","version":"150'876","reported_seq":4031,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.025990+0000","last_change":"2024-09-16T13:01:38.025990+0000","last_active":"2024-09-16T13:01:38.025990+0000","last_peered":"2024-09-16T13:01:38.025990+0000","last_clean":"2024-09-16T13:01:38.025990+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:01:38.025990+0000","last_undegraded":"2024-09-16T13:01:38.025990+0000","last_fullsized":"2024-09-16T13:01:38.025990+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:12.558678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.f","version":"149'919","reported_seq":36727,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.113134+0000","last_change":"2024-09-16T13:01:38.113695+0000","last_active":"2024-09-16T13:01:38.113134+0000","last_peered":"2024-09-16T13:01:38.113134+0000","last_clean":"2024-09-16T13:01:38.113134+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:01:38.113134+0000","last_undegraded":"2024-09-16T13:01:38.113134+0000","last_fullsized":"2024-09-16T13:01:38.113134+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1d"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":35,"num_objects":997,"num_object_clones":0,"num_object_copies":1994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":997,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21014,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":89,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":19194,"ondisk_log_size":19194,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192514,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301560,"kb_used_data":1760,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070280,"statfs":{"total":96636764160,"available":96327966720,"internally_reserved":0,"allocated":1802240,"data_stored":907367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303328,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068512,"statfs":{"total":96636764160,"available":96326156288,"internally_reserved":0,"allocated":1818624,"data_stored":914173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.224,"5min":1.272,"15min":1.272},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2130000000000001,"5min":1.391,"15min":1.391},"last":1.012}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":0.96899999999999997,"15min":0.89100000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.275,"5min":14.275,"15min":14.275},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.91100000000000003}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.999,"5min":1.095,"15min":1.111},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.097,"15min":1.075},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.0429999999999999}]},{"osd":4,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0720000000000001}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":1.137}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.1160000000000001,"15min":1.1499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4399999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.024,"5min":1.224,"15min":1.258},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.53,"5min":1.748,"15min":1.748},"last":1.2190000000000001}]}]},{"osd":4,"up_from":108,"seq":463856467978,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309192,"kb_used_data":1768,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062648,"statfs":{"total":96636764160,"available":96320151552,"internally_reserved":0,"allocated":1810432,"data_stored":914005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299144,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072696,"statfs":{"total":96636764160,"available":96330440704,"internally_reserved":0,"allocated":1830912,"data_stored":920449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.33900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.79400000000000004}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":1.091},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76900000000000002,"15min":0.77800000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.65400000000000003}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.82599999999999996,"15min":0.80500000000000005},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.752}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.278}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1319999999999999,"15min":1.1359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.097,"15min":1.115},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.3700000000000001}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":0.995},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.2370000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.113,"15min":1.1279999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.208,"15min":1.1919999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.1950000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299116,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072724,"statfs":{"total":96636764160,"available":96330469376,"internally_reserved":0,"allocated":1802240,"data_stored":913493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.80000000000000004,"15min":0.75900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.654,"5min":10.654,"15min":10.654},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.244,"5min":0.84599999999999997,"15min":0.78000000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.28799999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.88400000000000001,"15min":0.86199999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":0.90700000000000003,"15min":0.80900000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":18.117000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.88300000000000001,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.56999999999999995}]},{"osd":4,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.44800000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.159,"15min":1.2150000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.415,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.878,"5min":1.1180000000000001,"15min":1.1579999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5509999999999999,"5min":2.125,"15min":2.125},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.621}]},{"osd":7,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.2010000000000001,"15min":1.2490000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.792,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.2010000000000001,"15min":1.2609999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3899999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.63700000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":1888256,"data_stored":1516906,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":0.91300000000000003,"15min":0.85099999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.407,"5min":9.407,"15min":9.407},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.91100000000000003,"15min":0.96299999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.99399999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.123,"5min":2.254,"15min":2.254},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.80100000000000005,"15min":0.79300000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.3460000000000001,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.75},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.997,"15min":1.0429999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.256,"5min":2.044,"15min":2.044},"last":0.85599999999999998}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.0449999999999999,"15min":1.0740000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4370000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.1819999999999999,"15min":1.222},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.617,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.60999999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.96499999999999997}]},{"osd":7,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.163,"15min":1.1879999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6779999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.1319999999999999,"15min":1.167},"min":{"1min":0.438,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.583,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.878}]}]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303372,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068468,"statfs":{"total":96636764160,"available":96326111232,"internally_reserved":0,"allocated":1863680,"data_stored":1504354,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.03},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.2370000000000001,"15min":1.2909999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7470000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.125,"15min":1.1890000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.681,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.505},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.115,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.94899999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.071,"15min":1.069},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.2110000000000001,"15min":1.2250000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.147}]},{"osd":3,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0449999999999999,"15min":1.083},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2869999999999999,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.984},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.2649999999999999,"15min":1.319},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.623,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.9299999999999999}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.7889999999999997,"5min":4.7889999999999997,"15min":4.7889999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.82599999999999996}]},{"osd":5,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.284,"15min":1.339},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.75,"5min":6.282,"15min":6.282},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.96799999999999997,"15min":0.98799999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.611,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.89800000000000002}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.64400000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1810432,"data_stored":914005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85099999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.51500000000000001}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76600000000000001,"15min":0.78900000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.234,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.86199999999999999,"15min":0.88800000000000001},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.077,"5min":3.024,"15min":3.024},"last":0.68300000000000005}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":1.004,"15min":1.048},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.103,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.76800000000000002}]},{"osd":4,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.349,"15min":1.431},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.234,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.071,"15min":1.0880000000000001},"min":{"1min":0.59799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.79200000000000004}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.077}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1299999999999999,"15min":1.147},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.997},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.27,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.1140000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":51,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:01:41.720 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T13:01:41.721 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T13:01:41.721 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T13:01:41.721 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T13:01:42.016 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 154, 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7", 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-16T12:58:33.966359+0000", 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-16T13:01:41.032575+0000", 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-16T13:01:33.961419+0000", 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-16T12:59:39.968761+0000", 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5806080, 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-16T13:01:42.017 INFO:teuthology.orchestra.run.smithi042.stdout: "noscrub", 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 7, 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 4, 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "luminous", 2024-09-16T13:01:42.018 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "quincy", 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:58:44.284177+0000", 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T13:01:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T13:01:42.020 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "17", 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-16T13:01:42.021 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T13:01:42.022 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T13:01:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-16T13:01:42.024 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:58:51.733915+0000", 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T13:01:42.025 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T13:01:42.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "26", 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "26", 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 3, 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 24, 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T13:01:42.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [ 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: 3 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": 3, 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": 3, 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T13:01:42.028 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T13:01:42.029 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "cache", 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:59:00.193651+0000", 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8201, 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,incomplete_clones,selfmanaged_snaps", 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T13:01:42.030 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 4, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 4, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 4, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 4, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 4, 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T13:01:42.031 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "30", 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "26", 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 3, 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 24, 2024-09-16T13:01:42.032 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": 2, 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "writeback", 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 250, 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T13:01:42.033 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bloom", 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "false_positive_probability": 0.050000000000000003, 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "target_size": 0, 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "seed": 0 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 60, 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 8, 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T13:01:42.034 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [ 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: 1000000 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T13:01:42.035 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 4, 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-16T12:59:09.779825+0000", 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-16T13:01:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-16T13:01:42.037 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "151", 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 79, 2024-09-16T13:01:42.038 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 151, 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T13:01:42.039 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-16T13:01:42.040 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-16T13:01:42.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "0d8a8277-07cd-40f1-8c98-98b8b51d1689", 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 108, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:01:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:01:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22186 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6819/22186", 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6823/22186", 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6830/22186", 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6826/22186", 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "6ce5996a-5e0e-4c9d-b268-27b34a14d830", 2024-09-16T13:01:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 147, 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22184 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/22184", 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/22184", 2024-09-16T13:01:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/22184", 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/22184", 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "156042ab-909e-4368-b510-84018cb4354f", 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:01:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 121, 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-16T13:01:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-16T13:01:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22185 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/22185", 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/22185", 2024-09-16T13:01:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/22185", 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/22185", 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b6c53823-446f-4413-a2ff-d2ebe8fe9c71", 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:01:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 112, 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:01:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-16T13:01:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-16T13:01:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22187 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6818/22187", 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6822/22187", 2024-09-16T13:01:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/22187", 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6827/22187", 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "278bfad1-010b-494e-ba61-bdf3935ca078", 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:01:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 15, 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 83, 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 108, 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 147, 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 84, 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6816", 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 33598 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6817", 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 33598 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6818", 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 33598 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6819", 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 33598 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6822", 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 33598 2024-09-16T13:01:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6823", 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 33598 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6820", 2024-09-16T13:01:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 33598 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6821", 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 33598 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6817/33598", 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6819/33598", 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6823/33598", 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6821/33598", 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:01:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "ed1af73b-b108-4189-8223-f017e634e3e8", 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-16T13:01:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 121, 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6808", 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6809", 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:01:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:01:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6810", 2024-09-16T13:01:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:01:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6811", 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6814", 2024-09-16T13:01:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6815", 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6812", 2024-09-16T13:01:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6813", 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22340 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6809/22340", 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6811/22340", 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6815/22340", 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6813/22340", 2024-09-16T13:01:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "ce904ba8-02e9-4579-9635-b4b6c6de1a08", 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 15, 2024-09-16T13:01:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 120, 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 147, 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 147, 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 121, 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6824", 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 41077 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6825", 2024-09-16T13:01:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 41077 2024-09-16T13:01:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:01:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6826", 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 41077 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6827", 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 41077 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6830", 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 41077 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6831", 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 41077 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6828", 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 41077 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6829", 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 41077 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6825/41077", 2024-09-16T13:01:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6827/41077", 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6831/41077", 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6829/41077", 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d911da1b-2725-46a8-bbfa-8968de47271f", 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-16T13:01:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 121, 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6800", 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:01:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6801", 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6802", 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6803", 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6806", 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6807", 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-16T13:01:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6804", 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.136:6805", 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 22341 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.136:6801/22341", 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.136:6803/22341", 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.136:6807/22341", 2024-09-16T13:01:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.136:6805/22341", 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:01:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.264463+0000", 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.214474+0000", 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.217927+0000", 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:01:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.435822+0000", 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:01:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.451286+0000", 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 84 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.287403+0000", 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.440296+0000", 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 121 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-16T13:01:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540138320759226367, 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-16T12:58:41.140630+0000", 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [ 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "4.12", 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: 5, 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: 1 2024-09-16T13:01:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [ 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "4.4", 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mappings": [ 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "from": 1, 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "to": 6 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [ 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 4, 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 4, 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 29 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-16T13:01:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-16T13:01:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-16T13:01:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-16T13:01:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-16T13:01:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-16T13:01:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-16T13:01:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-16T13:01:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-16T13:01:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T13:01:42.099 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:01:42.101 INFO:tasks.thrashosds.thrasher:j is {'epoch': 154, 'fsid': 'c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7', 'created': '2024-09-16T12:58:33.966359+0000', 'modified': '2024-09-16T13:01:41.032575+0000', 'last_up_change': '2024-09-16T13:01:33.961419+0000', 'last_in_change': '2024-09-16T12:59:39.968761+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T12:58:44.284177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T12:58:51.733915+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T12:59:00.193651+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T12:59:09.779825+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 79, 'snap_epoch': 151, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': '0d8a8277-07cd-40f1-8c98-98b8b51d1689', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 108, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6817', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 22186}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6821', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 22186}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6830', 'nonce': 22186}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6825', 'nonce': 22186}, {'type': 'v1', 'addr': '172.21.15.42:6826', 'nonce': 22186}]}, 'public_addr': '172.21.15.42:6819/22186', 'cluster_addr': '172.21.15.42:6823/22186', 'heartbeat_back_addr': '172.21.15.42:6830/22186', 'heartbeat_front_addr': '172.21.15.42:6826/22186', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6ce5996a-5e0e-4c9d-b268-27b34a14d830', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 147, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 22184}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 22184}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 22184}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 22184}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 22184}]}, 'public_addr': '172.21.15.42:6809/22184', 'cluster_addr': '172.21.15.42:6811/22184', 'heartbeat_back_addr': '172.21.15.42:6815/22184', 'heartbeat_front_addr': '172.21.15.42:6813/22184', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '156042ab-909e-4368-b510-84018cb4354f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 22185}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 22185}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 22185}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 22185}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 22185}]}, 'public_addr': '172.21.15.42:6801/22185', 'cluster_addr': '172.21.15.42:6803/22185', 'heartbeat_back_addr': '172.21.15.42:6807/22185', 'heartbeat_front_addr': '172.21.15.42:6805/22185', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b6c53823-446f-4413-a2ff-d2ebe8fe9c71', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6818', 'nonce': 22187}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6822', 'nonce': 22187}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6829', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 22187}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 22187}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 22187}]}, 'public_addr': '172.21.15.42:6818/22187', 'cluster_addr': '172.21.15.42:6822/22187', 'heartbeat_back_addr': '172.21.15.42:6831/22187', 'heartbeat_front_addr': '172.21.15.42:6827/22187', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '278bfad1-010b-494e-ba61-bdf3935ca078', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 83, 'up_from': 108, 'up_thru': 147, 'down_at': 84, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6816', 'nonce': 33598}, {'type': 'v1', 'addr': '172.21.15.136:6817', 'nonce': 33598}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6818', 'nonce': 33598}, {'type': 'v1', 'addr': '172.21.15.136:6819', 'nonce': 33598}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6822', 'nonce': 33598}, {'type': 'v1', 'addr': '172.21.15.136:6823', 'nonce': 33598}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6820', 'nonce': 33598}, {'type': 'v1', 'addr': '172.21.15.136:6821', 'nonce': 33598}]}, 'public_addr': '172.21.15.136:6817/33598', 'cluster_addr': '172.21.15.136:6819/33598', 'heartbeat_back_addr': '172.21.15.136:6823/33598', 'heartbeat_front_addr': '172.21.15.136:6821/33598', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ed1af73b-b108-4189-8223-f017e634e3e8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6809', 'nonce': 22340}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6810', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6811', 'nonce': 22340}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6814', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6815', 'nonce': 22340}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 22340}, {'type': 'v1', 'addr': '172.21.15.136:6813', 'nonce': 22340}]}, 'public_addr': '172.21.15.136:6809/22340', 'cluster_addr': '172.21.15.136:6811/22340', 'heartbeat_back_addr': '172.21.15.136:6815/22340', 'heartbeat_front_addr': '172.21.15.136:6813/22340', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ce904ba8-02e9-4579-9635-b4b6c6de1a08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 120, 'up_from': 147, 'up_thru': 147, 'down_at': 121, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6824', 'nonce': 41077}, {'type': 'v1', 'addr': '172.21.15.136:6825', 'nonce': 41077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6826', 'nonce': 41077}, {'type': 'v1', 'addr': '172.21.15.136:6827', 'nonce': 41077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6830', 'nonce': 41077}, {'type': 'v1', 'addr': '172.21.15.136:6831', 'nonce': 41077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6828', 'nonce': 41077}, {'type': 'v1', 'addr': '172.21.15.136:6829', 'nonce': 41077}]}, 'public_addr': '172.21.15.136:6825/41077', 'cluster_addr': '172.21.15.136:6827/41077', 'heartbeat_back_addr': '172.21.15.136:6831/41077', 'heartbeat_front_addr': '172.21.15.136:6829/41077', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd911da1b-2725-46a8-bbfa-8968de47271f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6801', 'nonce': 22341}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6803', 'nonce': 22341}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6807', 'nonce': 22341}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 22341}, {'type': 'v1', 'addr': '172.21.15.136:6805', 'nonce': 22341}]}, 'public_addr': '172.21.15.136:6801/22341', 'cluster_addr': '172.21.15.136:6803/22341', 'heartbeat_back_addr': '172.21.15.136:6807/22341', 'heartbeat_front_addr': '172.21.15.136:6805/22341', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.264463+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.214474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.217927+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.435822+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.451286+0000', 'dead_epoch': 84}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.287403+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.440296+0000', 'dead_epoch': 121}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T12:58:41.140630+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '4.12', 'osds': [5, 1]}], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 1, 'to': 6}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 4, 'snaps': [{'begin': 4, 'length': 29}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T13:01:42.101 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:01:42.102 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-16T13:01:42.102 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T13:01:42.403 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:01:42.405 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:01:42.418 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-16T13:01:42.282854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590590,"num_objects":1006,"num_object_clones":0,"num_object_copies":2012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1001,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21119,"num_write_kb":309526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19274,"ondisk_log_size":19274,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2422364,"kb_used_data":14284,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752552356,"statfs":{"total":773094113280,"available":770613612544,"internally_reserved":0,"allocated":14626816,"data_stored":8504752,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52015452,"num_objects":-134,"num_object_clones":-49,"num_object_copies":-268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-135,"num_whiteouts":-32,"num_read":689,"num_read_kb":596,"num_write":161,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001303"},"pg_stats":[{"pgid":"4.1f","version":"150'338","reported_seq":4201,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.195460+0000","last_change":"2024-09-16T13:01:37.024157+0000","last_active":"2024-09-16T13:01:37.195460+0000","last_peered":"2024-09-16T13:01:37.195460+0000","last_clean":"2024-09-16T13:01:37.195460+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:01:37.195460+0000","last_undegraded":"2024-09-16T13:01:37.195460+0000","last_fullsized":"2024-09-16T13:01:37.195460+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:11.756487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002246297,"stat_sum":{"num_bytes":16,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":332,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"4.1e","version":"150'514","reported_seq":17213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027274+0000","last_change":"2024-09-16T13:01:38.062519+0000","last_active":"2024-09-16T13:01:39.027274+0000","last_peered":"2024-09-16T13:01:39.027274+0000","last_clean":"2024-09-16T13:01:39.027274+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:01:39.027274+0000","last_undegraded":"2024-09-16T13:01:39.027274+0000","last_fullsized":"2024-09-16T13:01:39.027274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1d","version":"142'433","reported_seq":4964,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.534114+0000","last_change":"2024-09-16T13:01:38.020672+0000","last_active":"2024-09-16T13:01:39.534114+0000","last_peered":"2024-09-16T13:01:39.534114+0000","last_clean":"2024-09-16T13:01:39.534114+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:01:39.534114+0000","last_undegraded":"2024-09-16T13:01:39.534114+0000","last_fullsized":"2024-09-16T13:01:39.534114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1c","version":"151'1171","reported_seq":7991,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755009+0000","last_change":"2024-09-16T13:01:38.017984+0000","last_active":"2024-09-16T13:01:39.755009+0000","last_peered":"2024-09-16T13:01:39.755009+0000","last_clean":"2024-09-16T13:01:39.755009+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:01:39.755009+0000","last_undegraded":"2024-09-16T13:01:39.755009+0000","last_fullsized":"2024-09-16T13:01:39.755009+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1b","version":"150'493","reported_seq":22019,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.026576+0000","last_change":"2024-09-16T13:01:38.026576+0000","last_active":"2024-09-16T13:01:38.026576+0000","last_peered":"2024-09-16T13:01:38.026576+0000","last_clean":"2024-09-16T13:01:38.026576+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:01:38.026576+0000","last_undegraded":"2024-09-16T13:01:38.026576+0000","last_fullsized":"2024-09-16T13:01:38.026576+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:04.244300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.1a","version":"150'685","reported_seq":7122,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.127606+0000","last_change":"2024-09-16T13:01:38.026337+0000","last_active":"2024-09-16T13:01:38.127606+0000","last_peered":"2024-09-16T13:01:38.127606+0000","last_clean":"2024-09-16T13:01:38.127606+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:01:38.127606+0000","last_undegraded":"2024-09-16T13:01:38.127606+0000","last_fullsized":"2024-09-16T13:01:38.127606+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:30.964357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.19","version":"149'399","reported_seq":5454,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533841+0000","last_change":"2024-09-16T13:01:38.021608+0000","last_active":"2024-09-16T13:01:39.533841+0000","last_peered":"2024-09-16T13:01:39.533841+0000","last_clean":"2024-09-16T13:01:39.533841+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:01:39.533841+0000","last_undegraded":"2024-09-16T13:01:39.533841+0000","last_fullsized":"2024-09-16T13:01:39.533841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.18","version":"150'440","reported_seq":6436,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.024730+0000","last_change":"2024-09-16T13:01:38.024730+0000","last_active":"2024-09-16T13:01:38.024730+0000","last_peered":"2024-09-16T13:01:38.024730+0000","last_clean":"2024-09-16T13:01:38.024730+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:01:38.024730+0000","last_undegraded":"2024-09-16T13:01:38.024730+0000","last_fullsized":"2024-09-16T13:01:38.024730+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:23.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.17","version":"150'889","reported_seq":7611,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.534295+0000","last_change":"2024-09-16T13:01:38.021503+0000","last_active":"2024-09-16T13:01:39.534295+0000","last_peered":"2024-09-16T13:01:39.534295+0000","last_clean":"2024-09-16T13:01:39.534295+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:01:39.534295+0000","last_undegraded":"2024-09-16T13:01:39.534295+0000","last_fullsized":"2024-09-16T13:01:39.534295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.16","version":"150'508","reported_seq":3289,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027463+0000","last_change":"2024-09-16T13:01:38.062149+0000","last_active":"2024-09-16T13:01:39.027463+0000","last_peered":"2024-09-16T13:01:39.027463+0000","last_clean":"2024-09-16T13:01:39.027463+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:01:39.027463+0000","last_undegraded":"2024-09-16T13:01:39.027463+0000","last_fullsized":"2024-09-16T13:01:39.027463+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.15","version":"150'523","reported_seq":16193,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755469+0000","last_change":"2024-09-16T13:01:38.021224+0000","last_active":"2024-09-16T13:01:39.755469+0000","last_peered":"2024-09-16T13:01:39.755469+0000","last_clean":"2024-09-16T13:01:39.755469+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:01:39.755469+0000","last_undegraded":"2024-09-16T13:01:39.755469+0000","last_fullsized":"2024-09-16T13:01:39.755469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.14","version":"149'579","reported_seq":23207,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.534166+0000","last_change":"2024-09-16T13:01:38.020686+0000","last_active":"2024-09-16T13:01:39.534166+0000","last_peered":"2024-09-16T13:01:39.534166+0000","last_clean":"2024-09-16T13:01:39.534166+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:01:39.534166+0000","last_undegraded":"2024-09-16T13:01:39.534166+0000","last_fullsized":"2024-09-16T13:01:39.534166+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.13","version":"151'512","reported_seq":6212,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.150928+0000","last_change":"2024-09-16T13:01:38.027017+0000","last_active":"2024-09-16T13:01:38.150928+0000","last_peered":"2024-09-16T13:01:38.150928+0000","last_clean":"2024-09-16T13:01:38.150928+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:01:38.150928+0000","last_undegraded":"2024-09-16T13:01:38.150928+0000","last_fullsized":"2024-09-16T13:01:38.150928+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:56.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.12","version":"150'371","reported_seq":3102,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.754993+0000","last_change":"2024-09-16T13:01:38.019785+0000","last_active":"2024-09-16T13:01:39.754993+0000","last_peered":"2024-09-16T13:01:39.754993+0000","last_clean":"2024-09-16T13:01:39.754993+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:01:39.754993+0000","last_undegraded":"2024-09-16T13:01:39.754993+0000","last_fullsized":"2024-09-16T13:01:39.754993+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.11","version":"149'363","reported_seq":18807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533789+0000","last_change":"2024-09-16T13:01:38.019836+0000","last_active":"2024-09-16T13:01:39.533789+0000","last_peered":"2024-09-16T13:01:39.533789+0000","last_clean":"2024-09-16T13:01:39.533789+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:01:39.533789+0000","last_undegraded":"2024-09-16T13:01:39.533789+0000","last_fullsized":"2024-09-16T13:01:39.533789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.10","version":"139'335","reported_seq":5438,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533789+0000","last_change":"2024-09-16T13:01:38.018514+0000","last_active":"2024-09-16T13:01:39.533789+0000","last_peered":"2024-09-16T13:01:39.533789+0000","last_clean":"2024-09-16T13:01:39.533789+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:01:39.533789+0000","last_undegraded":"2024-09-16T13:01:39.533789+0000","last_fullsized":"2024-09-16T13:01:39.533789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T13:01:36.009413+0000","last_change":"2024-09-16T13:01:35.025514+0000","last_active":"2024-09-16T13:01:36.009413+0000","last_peered":"2024-09-16T13:01:36.009413+0000","last_clean":"2024-09-16T13:01:36.009413+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:01:36.009413+0000","last_undegraded":"2024-09-16T13:01:36.009413+0000","last_fullsized":"2024-09-16T13:01:36.009413+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:27.902219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"146'451","reported_seq":3755,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.027021+0000","last_change":"2024-09-16T13:01:38.027021+0000","last_active":"2024-09-16T13:01:38.027021+0000","last_peered":"2024-09-16T13:01:38.027021+0000","last_clean":"2024-09-16T13:01:38.027021+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:01:38.027021+0000","last_undegraded":"2024-09-16T13:01:38.027021+0000","last_fullsized":"2024-09-16T13:01:38.027021+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.6","version":"0'0","reported_seq":117,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.114484+0000","last_change":"2024-09-16T13:01:34.980687+0000","last_active":"2024-09-16T13:01:37.114484+0000","last_peered":"2024-09-16T13:01:37.114484+0000","last_clean":"2024-09-16T13:01:37.114484+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:01:37.114484+0000","last_undegraded":"2024-09-16T13:01:37.114484+0000","last_fullsized":"2024-09-16T13:01:37.114484+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:44.616572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"149'690","reported_seq":7307,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533791+0000","last_change":"2024-09-16T13:01:38.020063+0000","last_active":"2024-09-16T13:01:39.533791+0000","last_peered":"2024-09-16T13:01:39.533791+0000","last_clean":"2024-09-16T13:01:39.533791+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:01:39.533791+0000","last_undegraded":"2024-09-16T13:01:39.533791+0000","last_fullsized":"2024-09-16T13:01:39.533791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.5","version":"0'0","reported_seq":144,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.166655+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:01:38.166655+0000","last_peered":"2024-09-16T13:01:38.166655+0000","last_clean":"2024-09-16T13:01:38.166655+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:01:38.166655+0000","last_undegraded":"2024-09-16T13:01:38.166655+0000","last_fullsized":"2024-09-16T13:01:38.166655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"151'687","reported_seq":3198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027087+0000","last_change":"2024-09-16T13:01:38.061542+0000","last_active":"2024-09-16T13:01:39.027087+0000","last_peered":"2024-09-16T13:01:39.027087+0000","last_clean":"2024-09-16T13:01:39.027087+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:01:39.027087+0000","last_undegraded":"2024-09-16T13:01:39.027087+0000","last_fullsized":"2024-09-16T13:01:39.027087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"2.4","version":"0'0","reported_seq":102,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.114188+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:01:37.114188+0000","last_peered":"2024-09-16T13:01:37.114188+0000","last_clean":"2024-09-16T13:01:37.114188+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:01:37.114188+0000","last_undegraded":"2024-09-16T13:01:37.114188+0000","last_fullsized":"2024-09-16T13:01:37.114188+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"150'586","reported_seq":17088,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.113796+0000","last_change":"2024-09-16T13:01:38.114806+0000","last_active":"2024-09-16T13:01:38.113796+0000","last_peered":"2024-09-16T13:01:38.113796+0000","last_clean":"2024-09-16T13:01:38.113796+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:01:38.113796+0000","last_undegraded":"2024-09-16T13:01:38.113796+0000","last_fullsized":"2024-09-16T13:01:38.113796+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"3.3","version":"0'0","reported_seq":106,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.023883+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:01:38.023883+0000","last_peered":"2024-09-16T13:01:38.023883+0000","last_clean":"2024-09-16T13:01:38.023883+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:01:38.023883+0000","last_undegraded":"2024-09-16T13:01:38.023883+0000","last_fullsized":"2024-09-16T13:01:38.023883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":172,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755578+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:01:39.755578+0000","last_peered":"2024-09-16T13:01:39.755578+0000","last_clean":"2024-09-16T13:01:39.755578+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:01:39.755578+0000","last_undegraded":"2024-09-16T13:01:39.755578+0000","last_fullsized":"2024-09-16T13:01:39.755578+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"148'501","reported_seq":17910,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.021958+0000","last_change":"2024-09-16T13:01:37.022816+0000","last_active":"2024-09-16T13:01:37.021958+0000","last_peered":"2024-09-16T13:01:37.021958+0000","last_clean":"2024-09-16T13:01:37.021958+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:01:37.021958+0000","last_undegraded":"2024-09-16T13:01:37.021958+0000","last_fullsized":"2024-09-16T13:01:37.021958+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:27.630481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010551410000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"3.0","version":"0'0","reported_seq":102,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.113765+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:01:37.113765+0000","last_peered":"2024-09-16T13:01:37.113765+0000","last_clean":"2024-09-16T13:01:37.113765+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:01:37.113765+0000","last_undegraded":"2024-09-16T13:01:37.113765+0000","last_fullsized":"2024-09-16T13:01:37.113765+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":147,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.754784+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:01:39.754784+0000","last_peered":"2024-09-16T13:01:39.754784+0000","last_clean":"2024-09-16T13:01:39.754784+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:01:39.754784+0000","last_undegraded":"2024-09-16T13:01:39.754784+0000","last_fullsized":"2024-09-16T13:01:39.754784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'407","reported_seq":2753,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.113285+0000","last_change":"2024-09-16T13:01:38.113780+0000","last_active":"2024-09-16T13:01:38.113285+0000","last_peered":"2024-09-16T13:01:38.113285+0000","last_clean":"2024-09-16T13:01:38.113285+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:01:38.113285+0000","last_undegraded":"2024-09-16T13:01:38.113285+0000","last_fullsized":"2024-09-16T13:01:38.113285+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"3.1","version":"55'1","reported_seq":150,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.026965+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:01:39.026965+0000","last_peered":"2024-09-16T13:01:39.026965+0000","last_clean":"2024-09-16T13:01:39.026965+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:01:39.026965+0000","last_undegraded":"2024-09-16T13:01:39.026965+0000","last_fullsized":"2024-09-16T13:01:39.026965+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.166779+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:01:38.166779+0000","last_peered":"2024-09-16T13:01:38.166779+0000","last_clean":"2024-09-16T13:01:38.166779+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:01:38.166779+0000","last_undegraded":"2024-09-16T13:01:38.166779+0000","last_fullsized":"2024-09-16T13:01:38.166779+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"151'434","reported_seq":7301,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027656+0000","last_change":"2024-09-16T13:01:38.062717+0000","last_active":"2024-09-16T13:01:39.027656+0000","last_peered":"2024-09-16T13:01:39.027656+0000","last_clean":"2024-09-16T13:01:39.027656+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:01:39.027656+0000","last_undegraded":"2024-09-16T13:01:39.027656+0000","last_fullsized":"2024-09-16T13:01:39.027656+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"3.2","version":"151'4","reported_seq":140,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.533958+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:01:39.533958+0000","last_peered":"2024-09-16T13:01:39.533958+0000","last_clean":"2024-09-16T13:01:39.533958+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:01:39.533958+0000","last_undegraded":"2024-09-16T13:01:39.533958+0000","last_fullsized":"2024-09-16T13:01:39.533958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":145,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.755376+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:01:39.755376+0000","last_peered":"2024-09-16T13:01:39.755376+0000","last_clean":"2024-09-16T13:01:39.755376+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:01:39.755376+0000","last_undegraded":"2024-09-16T13:01:39.755376+0000","last_fullsized":"2024-09-16T13:01:39.755376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":232,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.166950+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:01:38.166950+0000","last_peered":"2024-09-16T13:01:38.166950+0000","last_clean":"2024-09-16T13:01:38.166950+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:01:38.166950+0000","last_undegraded":"2024-09-16T13:01:38.166950+0000","last_fullsized":"2024-09-16T13:01:38.166950+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"150'444","reported_seq":3693,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.022639+0000","last_change":"2024-09-16T13:01:37.022639+0000","last_active":"2024-09-16T13:01:37.022639+0000","last_peered":"2024-09-16T13:01:37.022639+0000","last_clean":"2024-09-16T13:01:37.022639+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:01:37.022639+0000","last_undegraded":"2024-09-16T13:01:37.022639+0000","last_fullsized":"2024-09-16T13:01:37.022639+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:16.451880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090728,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"4.8","version":"150'977","reported_seq":18761,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.754914+0000","last_change":"2024-09-16T13:01:38.019797+0000","last_active":"2024-09-16T13:01:39.754914+0000","last_peered":"2024-09-16T13:01:39.754914+0000","last_clean":"2024-09-16T13:01:39.754914+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:01:39.754914+0000","last_undegraded":"2024-09-16T13:01:39.754914+0000","last_fullsized":"2024-09-16T13:01:39.754914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.9","version":"149'328","reported_seq":827,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.024649+0000","last_change":"2024-09-16T13:01:38.024649+0000","last_active":"2024-09-16T13:01:38.024649+0000","last_peered":"2024-09-16T13:01:38.024649+0000","last_clean":"2024-09-16T13:01:38.024649+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:01:38.024649+0000","last_undegraded":"2024-09-16T13:01:38.024649+0000","last_fullsized":"2024-09-16T13:01:38.024649+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.a","version":"150'1380","reported_seq":6945,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T13:01:37.023343+0000","last_change":"2024-09-16T13:01:37.023343+0000","last_active":"2024-09-16T13:01:37.023343+0000","last_peered":"2024-09-16T13:01:37.023343+0000","last_clean":"2024-09-16T13:01:37.023343+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:01:37.023343+0000","last_undegraded":"2024-09-16T13:01:37.023343+0000","last_fullsized":"2024-09-16T13:01:37.023343+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:00.052447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014800519999999999,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1c"}]},{"pgid":"4.b","version":"144'974","reported_seq":6616,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T13:01:39.027095+0000","last_change":"2024-09-16T13:01:38.060958+0000","last_active":"2024-09-16T13:01:39.027095+0000","last_peered":"2024-09-16T13:01:39.027095+0000","last_clean":"2024-09-16T13:01:39.027095+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:01:39.027095+0000","last_undegraded":"2024-09-16T13:01:39.027095+0000","last_fullsized":"2024-09-16T13:01:39.027095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.c","version":"151'553","reported_seq":5330,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.148180+0000","last_change":"2024-09-16T13:01:38.025883+0000","last_active":"2024-09-16T13:01:38.148180+0000","last_peered":"2024-09-16T13:01:38.148180+0000","last_clean":"2024-09-16T13:01:38.148180+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:01:38.148180+0000","last_undegraded":"2024-09-16T13:01:38.148180+0000","last_fullsized":"2024-09-16T13:01:38.148180+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.d","version":"150'434","reported_seq":4385,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.027791+0000","last_change":"2024-09-16T13:01:38.027791+0000","last_active":"2024-09-16T13:01:38.027791+0000","last_peered":"2024-09-16T13:01:38.027791+0000","last_clean":"2024-09-16T13:01:38.027791+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:01:38.027791+0000","last_undegraded":"2024-09-16T13:01:38.027791+0000","last_fullsized":"2024-09-16T13:01:38.027791+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.e","version":"150'876","reported_seq":4031,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.025990+0000","last_change":"2024-09-16T13:01:38.025990+0000","last_active":"2024-09-16T13:01:38.025990+0000","last_peered":"2024-09-16T13:01:38.025990+0000","last_clean":"2024-09-16T13:01:38.025990+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:01:38.025990+0000","last_undegraded":"2024-09-16T13:01:38.025990+0000","last_fullsized":"2024-09-16T13:01:38.025990+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:12.558678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1d"}]},{"pgid":"4.f","version":"149'919","reported_seq":36727,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-16T13:01:38.113134+0000","last_change":"2024-09-16T13:01:38.113695+0000","last_active":"2024-09-16T13:01:38.113134+0000","last_peered":"2024-09-16T13:01:38.113134+0000","last_clean":"2024-09-16T13:01:38.113134+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:01:38.113134+0000","last_undegraded":"2024-09-16T13:01:38.113134+0000","last_fullsized":"2024-09-16T13:01:38.113134+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1d"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":35,"num_objects":997,"num_object_clones":0,"num_object_copies":1994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":997,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21014,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":89,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":19194,"ondisk_log_size":19194,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192514,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301560,"kb_used_data":1760,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070280,"statfs":{"total":96636764160,"available":96327966720,"internally_reserved":0,"allocated":1802240,"data_stored":907367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303328,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068512,"statfs":{"total":96636764160,"available":96326156288,"internally_reserved":0,"allocated":1818624,"data_stored":914173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.224,"5min":1.272,"15min":1.272},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2130000000000001,"5min":1.391,"15min":1.391},"last":1.012}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":0.96899999999999997,"15min":0.89100000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.275,"5min":14.275,"15min":14.275},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.93400000000000005,"15min":0.90700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.7370000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.91100000000000003}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.90700000000000003,"15min":0.91400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.999,"5min":1.095,"15min":1.111},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.097,"15min":1.075},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.0429999999999999}]},{"osd":4,"last update":"Mon Sep 16 12:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0720000000000001}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":1.137}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.1160000000000001,"15min":1.1499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4399999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.024,"5min":1.224,"15min":1.258},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.53,"5min":1.748,"15min":1.748},"last":1.2190000000000001}]}]},{"osd":4,"up_from":108,"seq":463856467978,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309192,"kb_used_data":1768,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062648,"statfs":{"total":96636764160,"available":96320151552,"internally_reserved":0,"allocated":1810432,"data_stored":914005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299144,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072696,"statfs":{"total":96636764160,"available":96330440704,"internally_reserved":0,"allocated":1830912,"data_stored":920449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.33900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2549999999999999,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.79400000000000004}]},{"osd":1,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":1.091},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76900000000000002,"15min":0.77800000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.65400000000000003}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.82599999999999996,"15min":0.80500000000000005},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.752}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.278}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1319999999999999,"15min":1.1359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.097,"15min":1.115},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.3700000000000001}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":0.995},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.2370000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.113,"15min":1.1279999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.208,"15min":1.1919999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.1950000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299116,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072724,"statfs":{"total":96636764160,"available":96330469376,"internally_reserved":0,"allocated":1802240,"data_stored":913493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.80000000000000004,"15min":0.75900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.654,"5min":10.654,"15min":10.654},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.244,"5min":0.84599999999999997,"15min":0.78000000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.28799999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.88400000000000001,"15min":0.86199999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":0.90700000000000003,"15min":0.80900000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":18.117000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.88300000000000001,"15min":0.90400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.56999999999999995}]},{"osd":4,"last update":"Mon Sep 16 12:59:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.44800000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.159,"15min":1.2150000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.415,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.878,"5min":1.1180000000000001,"15min":1.1579999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5509999999999999,"5min":2.125,"15min":2.125},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.621}]},{"osd":7,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.2010000000000001,"15min":1.2490000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.792,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.2010000000000001,"15min":1.2609999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3899999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.63700000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":1888256,"data_stored":1516906,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":0.91300000000000003,"15min":0.85099999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.407,"5min":9.407,"15min":9.407},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.91100000000000003,"15min":0.96299999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.99399999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.123,"5min":2.254,"15min":2.254},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.80100000000000005,"15min":0.79300000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.3460000000000001,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.75},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.997,"15min":1.0429999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.256,"5min":2.044,"15min":2.044},"last":0.85599999999999998}]},{"osd":4,"last update":"Mon Sep 16 12:59:47 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.0449999999999999,"15min":1.0740000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4370000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.1819999999999999,"15min":1.222},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.617,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.60999999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.96499999999999997}]},{"osd":7,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.163,"15min":1.1879999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6779999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.1319999999999999,"15min":1.167},"min":{"1min":0.438,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.583,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.878}]}]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303372,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068468,"statfs":{"total":96636764160,"available":96326111232,"internally_reserved":0,"allocated":1863680,"data_stored":1504354,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.03},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.2370000000000001,"15min":1.2909999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7470000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.125,"15min":1.1890000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.681,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.505},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.115,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.94899999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.071,"15min":1.069},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.2110000000000001,"15min":1.2250000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":6.2290000000000001,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.147}]},{"osd":3,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0449999999999999,"15min":1.083},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2869999999999999,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.984},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.2649999999999999,"15min":1.319},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.623,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.9299999999999999}]},{"osd":4,"last update":"Mon Sep 16 12:59:46 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.7889999999999997,"5min":4.7889999999999997,"15min":4.7889999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.82599999999999996}]},{"osd":5,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.284,"15min":1.339},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.75,"5min":6.282,"15min":6.282},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.96799999999999997,"15min":0.98799999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.611,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.89800000000000002}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.64400000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1810432,"data_stored":914005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85099999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.51500000000000001}]},{"osd":2,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76600000000000001,"15min":0.78900000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.234,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.86199999999999999,"15min":0.88800000000000001},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.077,"5min":3.024,"15min":3.024},"last":0.68300000000000005}]},{"osd":3,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":1.004,"15min":1.048},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.103,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.76800000000000002}]},{"osd":4,"last update":"Mon Sep 16 12:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.349,"15min":1.431},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.234,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.071,"15min":1.0880000000000001},"min":{"1min":0.59799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.79200000000000004}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.077}]},{"osd":7,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1299999999999999,"15min":1.147},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.997},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.27,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.1140000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":51,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:01:42.421 INFO:tasks.thrashosds.thrasher:Setting 4.13 to [7, 1, 2, 5] 2024-09-16T13:01:42.421 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.13', '7', '1', '2', '5'] 2024-09-16T13:01:42.421 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 4.13 7 1 2 5 2024-09-16T13:01:43.074 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-16T13:01:43.074 INFO:teuthology.orchestra.run.smithi042.stderr:set 4.13 pg_upmap_items mapping to [7->1,2->5] 2024-09-16T13:01:48.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-16T13:01:48.230 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a: {} 2024-09-16T13:01:48.244 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b: {} 2024-09-16T13:01:48.258 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c: {} 2024-09-16T13:01:48.261 INFO:teuthology.orchestra.run.smithi042.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T13:01:48.261 INFO:teuthology.orchestra.run.smithi042.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T13:01:48.261 INFO:teuthology.orchestra.run.smithi042.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T13:01:48.275 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T13:01:48.712 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T13:01:48.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-16T13:01:49.715 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-16T13:01:49.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T13:01:50.719 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T13:01:50.731 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-16T13:01:51.731 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-16T13:01:51.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T13:01:52.744 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T13:01:52.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T13:01:53.748 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T13:01:53.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T13:01:54.765 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T13:01:54.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T13:01:55.768 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T13:01:55.782 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T13:01:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T13:01:56.800 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-16T13:01:57.789 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T13:01:57.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T13:01:58.803 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T13:01:58.817 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-16T13:01:59.812 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-16T13:01:59.826 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T13:02:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T13:02:00.828 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-16T13:02:01.827 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-16T13:02:01.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T13:02:02.837 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T13:02:02.851 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-16T13:02:03.852 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T13:02:03.863 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-16T13:02:03.863 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-16T13:02:03.863 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-16T13:02:03.863 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-16T13:02:03.864 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T13:02:03.864 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T13:02:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:04.156 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":173,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T13:02:03.831869+0000","last_up_change":"2024-09-16T13:01:33.961419+0000","last_in_change":"2024-09-16T12:59:39.968761+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T12:58:44.284177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T12:58:51.733915+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T12:59:00.193651+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T12:59:09.779825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":79,"snap_epoch":151,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":22186},{"type":"v1","addr":"172.21.15.42:6819","nonce":22186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":22186},{"type":"v1","addr":"172.21.15.42:6823","nonce":22186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":22186},{"type":"v1","addr":"172.21.15.42:6830","nonce":22186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":22186},{"type":"v1","addr":"172.21.15.42:6826","nonce":22186}]},"public_addr":"172.21.15.42:6819/22186","cluster_addr":"172.21.15.42:6823/22186","heartbeat_back_addr":"172.21.15.42:6830/22186","heartbeat_front_addr":"172.21.15.42:6826/22186","state":["exists","up"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":22184},{"type":"v1","addr":"172.21.15.42:6809","nonce":22184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":22184},{"type":"v1","addr":"172.21.15.42:6811","nonce":22184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":22184},{"type":"v1","addr":"172.21.15.42:6815","nonce":22184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":22184},{"type":"v1","addr":"172.21.15.42:6813","nonce":22184}]},"public_addr":"172.21.15.42:6809/22184","cluster_addr":"172.21.15.42:6811/22184","heartbeat_back_addr":"172.21.15.42:6815/22184","heartbeat_front_addr":"172.21.15.42:6813/22184","state":["exists","up"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":22185},{"type":"v1","addr":"172.21.15.42:6801","nonce":22185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":22185},{"type":"v1","addr":"172.21.15.42:6803","nonce":22185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":22185},{"type":"v1","addr":"172.21.15.42:6807","nonce":22185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":22185},{"type":"v1","addr":"172.21.15.42:6805","nonce":22185}]},"public_addr":"172.21.15.42:6801/22185","cluster_addr":"172.21.15.42:6803/22185","heartbeat_back_addr":"172.21.15.42:6807/22185","heartbeat_front_addr":"172.21.15.42:6805/22185","state":["exists","up"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":22187},{"type":"v1","addr":"172.21.15.42:6818","nonce":22187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":22187},{"type":"v1","addr":"172.21.15.42:6822","nonce":22187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":22187},{"type":"v1","addr":"172.21.15.42:6831","nonce":22187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":22187},{"type":"v1","addr":"172.21.15.42:6827","nonce":22187}]},"public_addr":"172.21.15.42:6818/22187","cluster_addr":"172.21.15.42:6822/22187","heartbeat_back_addr":"172.21.15.42:6831/22187","heartbeat_front_addr":"172.21.15.42:6827/22187","state":["exists","up"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":83,"up_from":108,"up_thru":147,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":33598},{"type":"v1","addr":"172.21.15.136:6817","nonce":33598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":33598},{"type":"v1","addr":"172.21.15.136:6819","nonce":33598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":33598},{"type":"v1","addr":"172.21.15.136:6823","nonce":33598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":33598},{"type":"v1","addr":"172.21.15.136:6821","nonce":33598}]},"public_addr":"172.21.15.136:6817/33598","cluster_addr":"172.21.15.136:6819/33598","heartbeat_back_addr":"172.21.15.136:6823/33598","heartbeat_front_addr":"172.21.15.136:6821/33598","state":["exists","up"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22340},{"type":"v1","addr":"172.21.15.136:6809","nonce":22340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22340},{"type":"v1","addr":"172.21.15.136:6811","nonce":22340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22340},{"type":"v1","addr":"172.21.15.136:6815","nonce":22340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22340},{"type":"v1","addr":"172.21.15.136:6813","nonce":22340}]},"public_addr":"172.21.15.136:6809/22340","cluster_addr":"172.21.15.136:6811/22340","heartbeat_back_addr":"172.21.15.136:6815/22340","heartbeat_front_addr":"172.21.15.136:6813/22340","state":["exists","up"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":120,"up_from":147,"up_thru":147,"down_at":121,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":41077},{"type":"v1","addr":"172.21.15.136:6825","nonce":41077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":41077},{"type":"v1","addr":"172.21.15.136:6827","nonce":41077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":41077},{"type":"v1","addr":"172.21.15.136:6831","nonce":41077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":41077},{"type":"v1","addr":"172.21.15.136:6829","nonce":41077}]},"public_addr":"172.21.15.136:6825/41077","cluster_addr":"172.21.15.136:6827/41077","heartbeat_back_addr":"172.21.15.136:6831/41077","heartbeat_front_addr":"172.21.15.136:6829/41077","state":["exists","up"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22341},{"type":"v1","addr":"172.21.15.136:6801","nonce":22341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22341},{"type":"v1","addr":"172.21.15.136:6803","nonce":22341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22341},{"type":"v1","addr":"172.21.15.136:6807","nonce":22341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22341},{"type":"v1","addr":"172.21.15.136:6805","nonce":22341}]},"public_addr":"172.21.15.136:6801/22341","cluster_addr":"172.21.15.136:6803/22341","heartbeat_back_addr":"172.21.15.136:6807/22341","heartbeat_front_addr":"172.21.15.136:6805/22341","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.264463+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.214474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.217927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.435822+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.451286+0000","dead_epoch":84},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.287403+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.440296+0000","dead_epoch":121},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.140630+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"4.12","osds":[5,1]}],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":1,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T13:02:04.175 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T13:02:04.175 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T13:02:04.471 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:04.471 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":173,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T13:02:03.831869+0000","last_up_change":"2024-09-16T13:01:33.961419+0000","last_in_change":"2024-09-16T12:59:39.968761+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T12:58:44.284177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T12:58:51.733915+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T12:59:00.193651+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T12:59:09.779825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":79,"snap_epoch":151,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":22186},{"type":"v1","addr":"172.21.15.42:6819","nonce":22186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":22186},{"type":"v1","addr":"172.21.15.42:6823","nonce":22186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":22186},{"type":"v1","addr":"172.21.15.42:6830","nonce":22186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":22186},{"type":"v1","addr":"172.21.15.42:6826","nonce":22186}]},"public_addr":"172.21.15.42:6819/22186","cluster_addr":"172.21.15.42:6823/22186","heartbeat_back_addr":"172.21.15.42:6830/22186","heartbeat_front_addr":"172.21.15.42:6826/22186","state":["exists","up"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":22184},{"type":"v1","addr":"172.21.15.42:6809","nonce":22184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":22184},{"type":"v1","addr":"172.21.15.42:6811","nonce":22184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":22184},{"type":"v1","addr":"172.21.15.42:6815","nonce":22184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":22184},{"type":"v1","addr":"172.21.15.42:6813","nonce":22184}]},"public_addr":"172.21.15.42:6809/22184","cluster_addr":"172.21.15.42:6811/22184","heartbeat_back_addr":"172.21.15.42:6815/22184","heartbeat_front_addr":"172.21.15.42:6813/22184","state":["exists","up"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":22185},{"type":"v1","addr":"172.21.15.42:6801","nonce":22185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":22185},{"type":"v1","addr":"172.21.15.42:6803","nonce":22185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":22185},{"type":"v1","addr":"172.21.15.42:6807","nonce":22185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":22185},{"type":"v1","addr":"172.21.15.42:6805","nonce":22185}]},"public_addr":"172.21.15.42:6801/22185","cluster_addr":"172.21.15.42:6803/22185","heartbeat_back_addr":"172.21.15.42:6807/22185","heartbeat_front_addr":"172.21.15.42:6805/22185","state":["exists","up"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":22187},{"type":"v1","addr":"172.21.15.42:6818","nonce":22187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":22187},{"type":"v1","addr":"172.21.15.42:6822","nonce":22187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":22187},{"type":"v1","addr":"172.21.15.42:6831","nonce":22187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":22187},{"type":"v1","addr":"172.21.15.42:6827","nonce":22187}]},"public_addr":"172.21.15.42:6818/22187","cluster_addr":"172.21.15.42:6822/22187","heartbeat_back_addr":"172.21.15.42:6831/22187","heartbeat_front_addr":"172.21.15.42:6827/22187","state":["exists","up"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":83,"up_from":108,"up_thru":147,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":33598},{"type":"v1","addr":"172.21.15.136:6817","nonce":33598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":33598},{"type":"v1","addr":"172.21.15.136:6819","nonce":33598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":33598},{"type":"v1","addr":"172.21.15.136:6823","nonce":33598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":33598},{"type":"v1","addr":"172.21.15.136:6821","nonce":33598}]},"public_addr":"172.21.15.136:6817/33598","cluster_addr":"172.21.15.136:6819/33598","heartbeat_back_addr":"172.21.15.136:6823/33598","heartbeat_front_addr":"172.21.15.136:6821/33598","state":["exists","up"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22340},{"type":"v1","addr":"172.21.15.136:6809","nonce":22340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22340},{"type":"v1","addr":"172.21.15.136:6811","nonce":22340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22340},{"type":"v1","addr":"172.21.15.136:6815","nonce":22340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22340},{"type":"v1","addr":"172.21.15.136:6813","nonce":22340}]},"public_addr":"172.21.15.136:6809/22340","cluster_addr":"172.21.15.136:6811/22340","heartbeat_back_addr":"172.21.15.136:6815/22340","heartbeat_front_addr":"172.21.15.136:6813/22340","state":["exists","up"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":120,"up_from":147,"up_thru":147,"down_at":121,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":41077},{"type":"v1","addr":"172.21.15.136:6825","nonce":41077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":41077},{"type":"v1","addr":"172.21.15.136:6827","nonce":41077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":41077},{"type":"v1","addr":"172.21.15.136:6831","nonce":41077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":41077},{"type":"v1","addr":"172.21.15.136:6829","nonce":41077}]},"public_addr":"172.21.15.136:6825/41077","cluster_addr":"172.21.15.136:6827/41077","heartbeat_back_addr":"172.21.15.136:6831/41077","heartbeat_front_addr":"172.21.15.136:6829/41077","state":["exists","up"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22341},{"type":"v1","addr":"172.21.15.136:6801","nonce":22341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22341},{"type":"v1","addr":"172.21.15.136:6803","nonce":22341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22341},{"type":"v1","addr":"172.21.15.136:6807","nonce":22341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22341},{"type":"v1","addr":"172.21.15.136:6805","nonce":22341}]},"public_addr":"172.21.15.136:6801/22341","cluster_addr":"172.21.15.136:6803/22341","heartbeat_back_addr":"172.21.15.136:6807/22341","heartbeat_front_addr":"172.21.15.136:6805/22341","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.264463+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.214474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.217927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.435822+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.451286+0000","dead_epoch":84},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.287403+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.440296+0000","dead_epoch":121},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.140630+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"4.12","osds":[5,1]}],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":1,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T13:02:04.489 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T13:02:04.490 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T13:02:04.490 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T13:02:04.490 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T13:02:04.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T13:02:04.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T13:02:04.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T13:02:04.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T13:02:04.694 INFO:teuthology.orchestra.run.smithi042.stdout:64424509482 2024-09-16T13:02:04.695 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T13:02:04.697 INFO:teuthology.orchestra.run.smithi042.stdout:64424509484 2024-09-16T13:02:04.697 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T13:02:04.700 INFO:teuthology.orchestra.run.smithi042.stdout:60129542189 2024-09-16T13:02:04.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T13:02:04.714 INFO:teuthology.orchestra.run.smithi042.stdout:463856467984 2024-09-16T13:02:04.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T13:02:04.716 INFO:teuthology.orchestra.run.smithi042.stdout:60129542188 2024-09-16T13:02:04.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T13:02:04.725 INFO:teuthology.orchestra.run.smithi042.stdout:64424509485 2024-09-16T13:02:04.725 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T13:02:04.727 INFO:teuthology.orchestra.run.smithi042.stdout:64424509484 2024-09-16T13:02:04.727 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T13:02:04.728 INFO:teuthology.orchestra.run.smithi042.stdout:631360192520 2024-09-16T13:02:04.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T13:02:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:60129542187 2024-09-16T13:02:05.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542187 for osd.7 2024-09-16T13:02:05.180 INFO:teuthology.orchestra.run.smithi042.stdout:64424509481 2024-09-16T13:02:05.192 INFO:teuthology.orchestra.run.smithi042.stdout:64424509483 2024-09-16T13:02:05.197 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509482 got 64424509481 for osd.3 2024-09-16T13:02:05.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509483 for osd.2 2024-09-16T13:02:05.209 INFO:teuthology.orchestra.run.smithi042.stdout:463856467982 2024-09-16T13:02:05.209 INFO:teuthology.orchestra.run.smithi042.stdout:60129542186 2024-09-16T13:02:05.220 INFO:teuthology.orchestra.run.smithi042.stdout:64424509484 2024-09-16T13:02:05.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542188 got 60129542186 for osd.1 2024-09-16T13:02:05.225 INFO:tasks.ceph.ceph_manager.ceph:need seq 463856467984 got 463856467982 for osd.4 2024-09-16T13:02:05.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509485 got 64424509484 for osd.0 2024-09-16T13:02:05.233 INFO:teuthology.orchestra.run.smithi042.stdout:631360192519 2024-09-16T13:02:05.238 INFO:teuthology.orchestra.run.smithi042.stdout:64424509483 2024-09-16T13:02:05.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 631360192520 got 631360192519 for osd.6 2024-09-16T13:02:05.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509483 for osd.5 2024-09-16T13:02:06.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T13:02:06.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T13:02:06.209 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T13:02:06.223 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T13:02:06.226 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T13:02:06.232 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T13:02:06.247 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T13:02:06.251 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T13:02:06.610 INFO:teuthology.orchestra.run.smithi042.stdout:60129542187 2024-09-16T13:02:06.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542187 for osd.7 2024-09-16T13:02:06.652 INFO:teuthology.orchestra.run.smithi042.stdout:64424509481 2024-09-16T13:02:06.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509482 got 64424509481 for osd.3 2024-09-16T13:02:06.671 INFO:teuthology.orchestra.run.smithi042.stdout:64424509483 2024-09-16T13:02:06.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509483 for osd.2 2024-09-16T13:02:06.705 INFO:teuthology.orchestra.run.smithi042.stdout:60129542186 2024-09-16T13:02:06.716 INFO:teuthology.orchestra.run.smithi042.stdout:631360192519 2024-09-16T13:02:06.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542188 got 60129542186 for osd.1 2024-09-16T13:02:06.729 INFO:teuthology.orchestra.run.smithi042.stdout:463856467982 2024-09-16T13:02:06.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 631360192520 got 631360192519 for osd.6 2024-09-16T13:02:06.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 463856467984 got 463856467982 for osd.4 2024-09-16T13:02:06.765 INFO:teuthology.orchestra.run.smithi042.stdout:64424509484 2024-09-16T13:02:06.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509485 got 64424509484 for osd.0 2024-09-16T13:02:06.778 INFO:teuthology.orchestra.run.smithi042.stdout:64424509483 2024-09-16T13:02:06.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509483 for osd.5 2024-09-16T13:02:07.629 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T13:02:07.664 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T13:02:07.687 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T13:02:07.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T13:02:07.730 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T13:02:07.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T13:02:07.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T13:02:07.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T13:02:08.067 INFO:teuthology.orchestra.run.smithi042.stdout:60129542189 2024-09-16T13:02:08.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542189 for osd.7 2024-09-16T13:02:08.086 DEBUG:teuthology.parallel:result is None 2024-09-16T13:02:08.131 INFO:teuthology.orchestra.run.smithi042.stdout:64424509483 2024-09-16T13:02:08.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509482 got 64424509483 for osd.3 2024-09-16T13:02:08.148 DEBUG:teuthology.parallel:result is None 2024-09-16T13:02:08.160 INFO:teuthology.orchestra.run.smithi042.stdout:64424509485 2024-09-16T13:02:08.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509485 for osd.2 2024-09-16T13:02:08.172 DEBUG:teuthology.parallel:result is None 2024-09-16T13:02:08.176 INFO:teuthology.orchestra.run.smithi042.stdout:463856467984 2024-09-16T13:02:08.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 463856467984 got 463856467984 for osd.4 2024-09-16T13:02:08.188 DEBUG:teuthology.parallel:result is None 2024-09-16T13:02:08.200 INFO:teuthology.orchestra.run.smithi042.stdout:60129542188 2024-09-16T13:02:08.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542188 got 60129542188 for osd.1 2024-09-16T13:02:08.212 DEBUG:teuthology.parallel:result is None 2024-09-16T13:02:08.237 INFO:teuthology.orchestra.run.smithi042.stdout:631360192520 2024-09-16T13:02:08.244 INFO:teuthology.orchestra.run.smithi042.stdout:64424509486 2024-09-16T13:02:08.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 631360192520 got 631360192520 for osd.6 2024-09-16T13:02:08.249 DEBUG:teuthology.parallel:result is None 2024-09-16T13:02:08.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509485 got 64424509486 for osd.0 2024-09-16T13:02:08.256 DEBUG:teuthology.parallel:result is None 2024-09-16T13:02:08.262 INFO:teuthology.orchestra.run.smithi042.stdout:64424509485 2024-09-16T13:02:08.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509485 for osd.5 2024-09-16T13:02:08.275 DEBUG:teuthology.parallel:result is None 2024-09-16T13:02:08.275 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T13:02:08.275 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:02:08.564 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:08.566 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:02:08.577 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-16T13:02:08.290504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":1005,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21120,"num_write_kb":309526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19275,"ondisk_log_size":19275,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424272,"kb_used_data":16316,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752550448,"statfs":{"total":773094113280,"available":770611658752,"internally_reserved":0,"allocated":16707584,"data_stored":9569383,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.475073"},"pg_stats":[{"pgid":"4.1f","version":"151'339","reported_seq":4229,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865786+0000","last_change":"2024-09-16T13:01:38.032360+0000","last_active":"2024-09-16T13:02:02.865786+0000","last_peered":"2024-09-16T13:02:02.865786+0000","last_clean":"2024-09-16T13:02:02.865786+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:02:02.865786+0000","last_undegraded":"2024-09-16T13:02:02.865786+0000","last_fullsized":"2024-09-16T13:02:02.865786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:11.756487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015622850000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":333,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"150'514","reported_seq":17234,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856527+0000","last_change":"2024-09-16T13:01:38.062519+0000","last_active":"2024-09-16T13:02:02.856527+0000","last_peered":"2024-09-16T13:02:02.856527+0000","last_clean":"2024-09-16T13:02:02.856527+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:02:02.856527+0000","last_undegraded":"2024-09-16T13:02:02.856527+0000","last_fullsized":"2024-09-16T13:02:02.856527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"142'433","reported_seq":4984,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829498+0000","last_change":"2024-09-16T13:01:38.020672+0000","last_active":"2024-09-16T13:02:02.829498+0000","last_peered":"2024-09-16T13:02:02.829498+0000","last_clean":"2024-09-16T13:02:02.829498+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:02:02.829498+0000","last_undegraded":"2024-09-16T13:02:02.829498+0000","last_fullsized":"2024-09-16T13:02:02.829498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"151'1171","reported_seq":8013,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863870+0000","last_change":"2024-09-16T13:01:38.017984+0000","last_active":"2024-09-16T13:02:02.863870+0000","last_peered":"2024-09-16T13:02:02.863870+0000","last_clean":"2024-09-16T13:02:02.863870+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:02:02.863870+0000","last_undegraded":"2024-09-16T13:02:02.863870+0000","last_fullsized":"2024-09-16T13:02:02.863870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"150'493","reported_seq":22038,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817747+0000","last_change":"2024-09-16T13:01:38.026576+0000","last_active":"2024-09-16T13:02:01.817747+0000","last_peered":"2024-09-16T13:02:01.817747+0000","last_clean":"2024-09-16T13:02:01.817747+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:02:01.817747+0000","last_undegraded":"2024-09-16T13:02:01.817747+0000","last_fullsized":"2024-09-16T13:02:01.817747+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:04.244300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"150'685","reported_seq":7141,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817710+0000","last_change":"2024-09-16T13:01:38.026337+0000","last_active":"2024-09-16T13:02:01.817710+0000","last_peered":"2024-09-16T13:02:01.817710+0000","last_clean":"2024-09-16T13:02:01.817710+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:02:01.817710+0000","last_undegraded":"2024-09-16T13:02:01.817710+0000","last_fullsized":"2024-09-16T13:02:01.817710+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:30.964357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"149'399","reported_seq":5474,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828856+0000","last_change":"2024-09-16T13:01:38.021608+0000","last_active":"2024-09-16T13:02:02.828856+0000","last_peered":"2024-09-16T13:02:02.828856+0000","last_clean":"2024-09-16T13:02:02.828856+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:02:02.828856+0000","last_undegraded":"2024-09-16T13:02:02.828856+0000","last_fullsized":"2024-09-16T13:02:02.828856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"150'440","reported_seq":6455,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817051+0000","last_change":"2024-09-16T13:01:38.024730+0000","last_active":"2024-09-16T13:02:01.817051+0000","last_peered":"2024-09-16T13:02:01.817051+0000","last_clean":"2024-09-16T13:02:01.817051+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:02:01.817051+0000","last_undegraded":"2024-09-16T13:02:01.817051+0000","last_fullsized":"2024-09-16T13:02:01.817051+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:23.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"150'889","reported_seq":7631,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829608+0000","last_change":"2024-09-16T13:01:38.021503+0000","last_active":"2024-09-16T13:02:02.829608+0000","last_peered":"2024-09-16T13:02:02.829608+0000","last_clean":"2024-09-16T13:02:02.829608+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:02:02.829608+0000","last_undegraded":"2024-09-16T13:02:02.829608+0000","last_fullsized":"2024-09-16T13:02:02.829608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"150'508","reported_seq":3310,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856528+0000","last_change":"2024-09-16T13:01:38.062149+0000","last_active":"2024-09-16T13:02:02.856528+0000","last_peered":"2024-09-16T13:02:02.856528+0000","last_clean":"2024-09-16T13:02:02.856528+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:02:02.856528+0000","last_undegraded":"2024-09-16T13:02:02.856528+0000","last_fullsized":"2024-09-16T13:02:02.856528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"150'523","reported_seq":16214,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829042+0000","last_change":"2024-09-16T13:01:38.021224+0000","last_active":"2024-09-16T13:02:02.829042+0000","last_peered":"2024-09-16T13:02:02.829042+0000","last_clean":"2024-09-16T13:02:02.829042+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:02:02.829042+0000","last_undegraded":"2024-09-16T13:02:02.829042+0000","last_fullsized":"2024-09-16T13:02:02.829042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"149'579","reported_seq":23227,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829478+0000","last_change":"2024-09-16T13:01:38.020686+0000","last_active":"2024-09-16T13:02:02.829478+0000","last_peered":"2024-09-16T13:02:02.829478+0000","last_clean":"2024-09-16T13:02:02.829478+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:02:02.829478+0000","last_undegraded":"2024-09-16T13:02:02.829478+0000","last_fullsized":"2024-09-16T13:02:02.829478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"151'512","reported_seq":6231,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817958+0000","last_change":"2024-09-16T13:01:38.027017+0000","last_active":"2024-09-16T13:02:01.817958+0000","last_peered":"2024-09-16T13:02:01.817958+0000","last_clean":"2024-09-16T13:02:01.817958+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:02:01.817958+0000","last_undegraded":"2024-09-16T13:02:01.817958+0000","last_fullsized":"2024-09-16T13:02:01.817958+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:56.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"150'371","reported_seq":3123,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828778+0000","last_change":"2024-09-16T13:01:38.019785+0000","last_active":"2024-09-16T13:02:02.828778+0000","last_peered":"2024-09-16T13:02:02.828778+0000","last_clean":"2024-09-16T13:02:02.828778+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:02:02.828778+0000","last_undegraded":"2024-09-16T13:02:02.828778+0000","last_fullsized":"2024-09-16T13:02:02.828778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"149'363","reported_seq":18827,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828605+0000","last_change":"2024-09-16T13:01:38.019836+0000","last_active":"2024-09-16T13:02:02.828605+0000","last_peered":"2024-09-16T13:02:02.828605+0000","last_clean":"2024-09-16T13:02:02.828605+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:02:02.828605+0000","last_undegraded":"2024-09-16T13:02:02.828605+0000","last_fullsized":"2024-09-16T13:02:02.828605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"139'335","reported_seq":5458,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.018514+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865667+0000","last_change":"2024-09-16T13:01:35.025514+0000","last_active":"2024-09-16T13:02:02.865667+0000","last_peered":"2024-09-16T13:02:02.865667+0000","last_clean":"2024-09-16T13:02:02.865667+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:02:02.865667+0000","last_undegraded":"2024-09-16T13:02:02.865667+0000","last_fullsized":"2024-09-16T13:02:02.865667+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:27.902219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"146'451","reported_seq":3774,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817602+0000","last_change":"2024-09-16T13:01:38.027021+0000","last_active":"2024-09-16T13:02:01.817602+0000","last_peered":"2024-09-16T13:02:01.817602+0000","last_clean":"2024-09-16T13:02:01.817602+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:02:01.817602+0000","last_undegraded":"2024-09-16T13:02:01.817602+0000","last_fullsized":"2024-09-16T13:02:01.817602+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460996+0000","last_change":"2024-09-16T13:01:34.980687+0000","last_active":"2024-09-16T13:02:02.460996+0000","last_peered":"2024-09-16T13:02:02.460996+0000","last_clean":"2024-09-16T13:02:02.460996+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:02:02.460996+0000","last_undegraded":"2024-09-16T13:02:02.460996+0000","last_fullsized":"2024-09-16T13:02:02.460996+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:44.616572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"149'690","reported_seq":7327,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.020063+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":160,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459793+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:02:02.459793+0000","last_peered":"2024-09-16T13:02:02.459793+0000","last_clean":"2024-09-16T13:02:02.459793+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:02:02.459793+0000","last_undegraded":"2024-09-16T13:02:02.459793+0000","last_fullsized":"2024-09-16T13:02:02.459793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"151'687","reported_seq":3219,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856470+0000","last_change":"2024-09-16T13:01:38.061542+0000","last_active":"2024-09-16T13:02:02.856470+0000","last_peered":"2024-09-16T13:02:02.856470+0000","last_clean":"2024-09-16T13:02:02.856470+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:02:02.856470+0000","last_undegraded":"2024-09-16T13:02:02.856470+0000","last_fullsized":"2024-09-16T13:02:02.856470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460629+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:02:02.460629+0000","last_peered":"2024-09-16T13:02:02.460629+0000","last_clean":"2024-09-16T13:02:02.460629+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:02:02.460629+0000","last_undegraded":"2024-09-16T13:02:02.460629+0000","last_fullsized":"2024-09-16T13:02:02.460629+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"150'586","reported_seq":17109,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460622+0000","last_change":"2024-09-16T13:01:38.114806+0000","last_active":"2024-09-16T13:02:02.460622+0000","last_peered":"2024-09-16T13:02:02.460622+0000","last_clean":"2024-09-16T13:02:02.460622+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:02:02.460622+0000","last_undegraded":"2024-09-16T13:02:02.460622+0000","last_fullsized":"2024-09-16T13:02:02.460622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":123,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817718+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:02:01.817718+0000","last_peered":"2024-09-16T13:02:01.817718+0000","last_clean":"2024-09-16T13:02:01.817718+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:02:01.817718+0000","last_undegraded":"2024-09-16T13:02:01.817718+0000","last_fullsized":"2024-09-16T13:02:01.817718+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":191,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829038+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:02:02.829038+0000","last_peered":"2024-09-16T13:02:02.829038+0000","last_clean":"2024-09-16T13:02:02.829038+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:02:02.829038+0000","last_undegraded":"2024-09-16T13:02:02.829038+0000","last_fullsized":"2024-09-16T13:02:02.829038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"148'501","reported_seq":17937,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865403+0000","last_change":"2024-09-16T13:01:38.031901+0000","last_active":"2024-09-16T13:02:02.865403+0000","last_peered":"2024-09-16T13:02:02.865403+0000","last_clean":"2024-09-16T13:02:02.865403+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:02:02.865403+0000","last_undegraded":"2024-09-16T13:02:02.865403+0000","last_fullsized":"2024-09-16T13:02:02.865403+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:27.630481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411569999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":121,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460348+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:02:02.460348+0000","last_peered":"2024-09-16T13:02:02.460348+0000","last_clean":"2024-09-16T13:02:02.460348+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:02:02.460348+0000","last_undegraded":"2024-09-16T13:02:02.460348+0000","last_fullsized":"2024-09-16T13:02:02.460348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863815+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:02:02.863815+0000","last_peered":"2024-09-16T13:02:02.863815+0000","last_clean":"2024-09-16T13:02:02.863815+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:02:02.863815+0000","last_undegraded":"2024-09-16T13:02:02.863815+0000","last_fullsized":"2024-09-16T13:02:02.863815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'407","reported_seq":2774,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.461059+0000","last_change":"2024-09-16T13:01:38.113780+0000","last_active":"2024-09-16T13:02:02.461059+0000","last_peered":"2024-09-16T13:02:02.461059+0000","last_clean":"2024-09-16T13:02:02.461059+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:02:02.461059+0000","last_undegraded":"2024-09-16T13:02:02.461059+0000","last_fullsized":"2024-09-16T13:02:02.461059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":169,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856355+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:02:02.856355+0000","last_peered":"2024-09-16T13:02:02.856355+0000","last_clean":"2024-09-16T13:02:02.856355+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:02:02.856355+0000","last_undegraded":"2024-09-16T13:02:02.856355+0000","last_fullsized":"2024-09-16T13:02:02.856355+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459690+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:02:02.459690+0000","last_peered":"2024-09-16T13:02:02.459690+0000","last_clean":"2024-09-16T13:02:02.459690+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:02:02.459690+0000","last_undegraded":"2024-09-16T13:02:02.459690+0000","last_fullsized":"2024-09-16T13:02:02.459690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"151'434","reported_seq":7322,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856609+0000","last_change":"2024-09-16T13:01:38.062717+0000","last_active":"2024-09-16T13:02:02.856609+0000","last_peered":"2024-09-16T13:02:02.856609+0000","last_clean":"2024-09-16T13:02:02.856609+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:02:02.856609+0000","last_undegraded":"2024-09-16T13:02:02.856609+0000","last_fullsized":"2024-09-16T13:02:02.856609+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"151'4","reported_seq":158,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829055+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:02:02.829055+0000","last_peered":"2024-09-16T13:02:02.829055+0000","last_clean":"2024-09-16T13:02:02.829055+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:02:02.829055+0000","last_undegraded":"2024-09-16T13:02:02.829055+0000","last_fullsized":"2024-09-16T13:02:02.829055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":164,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829054+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:02:02.829054+0000","last_peered":"2024-09-16T13:02:02.829054+0000","last_clean":"2024-09-16T13:02:02.829054+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:02:02.829054+0000","last_undegraded":"2024-09-16T13:02:02.829054+0000","last_fullsized":"2024-09-16T13:02:02.829054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":248,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459761+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:02:02.459761+0000","last_peered":"2024-09-16T13:02:02.459761+0000","last_clean":"2024-09-16T13:02:02.459761+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:02:02.459761+0000","last_undegraded":"2024-09-16T13:02:02.459761+0000","last_fullsized":"2024-09-16T13:02:02.459761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"150'444","reported_seq":3720,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865541+0000","last_change":"2024-09-16T13:01:38.031044+0000","last_active":"2024-09-16T13:02:02.865541+0000","last_peered":"2024-09-16T13:02:02.865541+0000","last_clean":"2024-09-16T13:02:02.865541+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:02:02.865541+0000","last_undegraded":"2024-09-16T13:02:02.865541+0000","last_fullsized":"2024-09-16T13:02:02.865541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:16.451880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602565,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"150'977","reported_seq":18782,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828748+0000","last_change":"2024-09-16T13:01:38.019797+0000","last_active":"2024-09-16T13:02:02.828748+0000","last_peered":"2024-09-16T13:02:02.828748+0000","last_clean":"2024-09-16T13:02:02.828748+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:02:02.828748+0000","last_undegraded":"2024-09-16T13:02:02.828748+0000","last_fullsized":"2024-09-16T13:02:02.828748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"149'328","reported_seq":846,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817233+0000","last_change":"2024-09-16T13:01:38.024649+0000","last_active":"2024-09-16T13:02:01.817233+0000","last_peered":"2024-09-16T13:02:01.817233+0000","last_clean":"2024-09-16T13:02:01.817233+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:02:01.817233+0000","last_undegraded":"2024-09-16T13:02:01.817233+0000","last_fullsized":"2024-09-16T13:02:01.817233+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"150'1380","reported_seq":6972,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865373+0000","last_change":"2024-09-16T13:01:38.031686+0000","last_active":"2024-09-16T13:02:02.865373+0000","last_peered":"2024-09-16T13:02:02.865373+0000","last_clean":"2024-09-16T13:02:02.865373+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:02:02.865373+0000","last_undegraded":"2024-09-16T13:02:02.865373+0000","last_fullsized":"2024-09-16T13:02:02.865373+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:00.052447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093887699999999996,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"144'974","reported_seq":6637,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856424+0000","last_change":"2024-09-16T13:01:38.060958+0000","last_active":"2024-09-16T13:02:02.856424+0000","last_peered":"2024-09-16T13:02:02.856424+0000","last_clean":"2024-09-16T13:02:02.856424+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:02:02.856424+0000","last_undegraded":"2024-09-16T13:02:02.856424+0000","last_fullsized":"2024-09-16T13:02:02.856424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"151'553","reported_seq":5349,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817849+0000","last_change":"2024-09-16T13:01:38.025883+0000","last_active":"2024-09-16T13:02:01.817849+0000","last_peered":"2024-09-16T13:02:01.817849+0000","last_clean":"2024-09-16T13:02:01.817849+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:02:01.817849+0000","last_undegraded":"2024-09-16T13:02:01.817849+0000","last_fullsized":"2024-09-16T13:02:01.817849+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"150'434","reported_seq":4404,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817961+0000","last_change":"2024-09-16T13:01:38.027791+0000","last_active":"2024-09-16T13:02:01.817961+0000","last_peered":"2024-09-16T13:02:01.817961+0000","last_clean":"2024-09-16T13:02:01.817961+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:02:01.817961+0000","last_undegraded":"2024-09-16T13:02:01.817961+0000","last_fullsized":"2024-09-16T13:02:01.817961+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"150'876","reported_seq":4050,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.818018+0000","last_change":"2024-09-16T13:01:38.025990+0000","last_active":"2024-09-16T13:02:01.818018+0000","last_peered":"2024-09-16T13:02:01.818018+0000","last_clean":"2024-09-16T13:02:01.818018+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:02:01.818018+0000","last_undegraded":"2024-09-16T13:02:01.818018+0000","last_fullsized":"2024-09-16T13:02:01.818018+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:12.558678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"149'919","reported_seq":36748,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.461057+0000","last_change":"2024-09-16T13:01:38.113695+0000","last_active":"2024-09-16T13:02:02.461057+0000","last_peered":"2024-09-16T13:02:02.461057+0000","last_clean":"2024-09-16T13:02:02.461057+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:02:02.461057+0000","last_undegraded":"2024-09-16T13:02:02.461057+0000","last_fullsized":"2024-09-16T13:02:02.461057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21015,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":19195,"ondisk_log_size":19195,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192521,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301756,"kb_used_data":2028,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070084,"statfs":{"total":96636764160,"available":96327766016,"internally_reserved":0,"allocated":2076672,"data_stored":1051715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]}]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303568,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068272,"statfs":{"total":96636764160,"available":96325910528,"internally_reserved":0,"allocated":2088960,"data_stored":1051902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.85599999999999998},"min":{"1min":0.48099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1499999999999999,"5min":1.272,"15min":1.272},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.83999999999999997},"min":{"1min":0.55100000000000005,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.391,"15min":1.391},"last":0.57899999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.97299999999999998,"15min":0.89200000000000002},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.333,"5min":14.275,"15min":14.275},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.58699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3600000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.752}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.58099999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2430000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.996,"5min":1.071,"15min":1.103},"min":{"1min":0.61799999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4830000000000001,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.95299999999999996}]},{"osd":3,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.071,"15min":1.0669999999999999},"min":{"1min":0.61899999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.2989999999999999,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.115},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6299999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.272}]},{"osd":4,"last update":"Mon Sep 16 13:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0109999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":1.446},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.077,"15min":1.137},"min":{"1min":0.65300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4379999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.29},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1850000000000001,"15min":1.2450000000000001},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.673,"5min":1.748,"15min":1.748},"last":1.369}]}]},{"osd":4,"up_from":108,"seq":463856467984,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309436,"kb_used_data":2020,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062404,"statfs":{"total":96636764160,"available":96319901696,"internally_reserved":0,"allocated":2068480,"data_stored":1045486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.0309999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.3899999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.23},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.4630000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.4339999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]},{"osd":7,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.498}]}]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299392,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072448,"statfs":{"total":96636764160,"available":96330186752,"internally_reserved":0,"allocated":2076672,"data_stored":1045654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.995,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71799999999999997,"15min":0.66700000000000004},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0940000000000001,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.232,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.873},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.51500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.103,"5min":1.528,"15min":1.528},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98399999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.49299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5740000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.0529999999999999}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.165}]},{"osd":5,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.109,"15min":1.129},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8520000000000001,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.171,"5min":1.1060000000000001,"15min":1.1180000000000001},"min":{"1min":0.56000000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.143}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1299999999999999},"min":{"1min":0.61099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.363},{"interface":"front","average":{"1min":1.167,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.77700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9830000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.304}]}]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299388,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072452,"statfs":{"total":96636764160,"available":96330190848,"internally_reserved":0,"allocated":2072576,"data_stored":1051222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.314,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.0980000000000001,"5min":10.654,"15min":10.654},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.77100000000000002},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.97899999999999998,"5min":10.816000000000001,"15min":10.816000000000001},"last":1.2589999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.52200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99099999999999999,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.97999999999999998}]},{"osd":3,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.89200000000000002,"15min":0.80400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.2430000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.077,"5min":1.647,"15min":1.647},"last":0.873}]},{"osd":4,"last update":"Mon Sep 16 13:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.365}]},{"osd":5,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.093,"15min":1.1930000000000001},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.226,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.516},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0740000000000001,"15min":1.1439999999999999},"min":{"1min":0.60799999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.383,"5min":2.125,"15min":2.125},"last":1.772}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.8089999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.681}]},{"osd":7,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.139,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.478,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.7350000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.1539999999999999,"15min":1.2450000000000001},"min":{"1min":0.63700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.538,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.901}]}]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303600,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068240,"statfs":{"total":96636764160,"available":96325877760,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.88600000000000001,"15min":0.84199999999999997},"min":{"1min":0.39100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.1899999999999999,"5min":9.407,"15min":9.407},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.85999999999999999,"15min":0.94599999999999995},"min":{"1min":0.46899999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.319,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0469999999999999,"5min":2.254,"15min":2.254},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.36399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83199999999999996,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":1.028},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4750000000000001,"5min":2.044,"15min":2.044},"last":0.879}]},{"osd":4,"last update":"Mon Sep 16 13:01:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.038},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.177}]},{"osd":5,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0149999999999999,"15min":1.0640000000000001},"min":{"1min":0.56399999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4730000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.999,"5min":1.133,"15min":1.2050000000000001},"min":{"1min":0.60999999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.325,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.125}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.145}]},{"osd":7,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.129,"15min":1.177},"min":{"1min":0.745,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.327,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.093},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1599999999999999},"min":{"1min":0.70099999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.427,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.2010000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303596,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068244,"statfs":{"total":96636764160,"available":96325881856,"internally_reserved":0,"allocated":2121728,"data_stored":1635835,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5920000000000001,"5min":1.73,"15min":1.73},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.145,"15min":1.26},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3080000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.98199999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0249999999999999,"15min":1.1559999999999999},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.208,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.56499999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5149999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.81999999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0569999999999999},"min":{"1min":0.55400000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.298,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.143,"15min":1.2030000000000001},"min":{"1min":0.50900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1839999999999999,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.0189999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0109999999999999,"15min":1.0720000000000001},"min":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.984,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.117},{"interface":"front","average":{"1min":1.028,"5min":1.2010000000000001,"15min":1.298},"min":{"1min":0.66900000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9299999999999999,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.272}]},{"osd":4,"last update":"Mon Sep 16 13:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.046},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.96199999999999997}]},{"osd":5,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.1759999999999999,"15min":1.3029999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.135,"5min":6.282,"15min":6.282},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.93600000000000005,"15min":0.97799999999999998},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.2549999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.153}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.083},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.349}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303536,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068304,"statfs":{"total":96636764160,"available":96325943296,"internally_reserved":0,"allocated":2068480,"data_stored":1045486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.73999999999999999},"min":{"1min":0.38100000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.97799999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.254,"15min":0.254},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72799999999999998,"15min":0.77600000000000002},"min":{"1min":0.35699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.90400000000000003,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.181,"5min":3.024,"15min":3.024},"last":1.1990000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.95999999999999996,"15min":1.034},"min":{"1min":0.53600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2250000000000001,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.49199999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.151,"5min":3.544,"15min":3.544},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.034},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.25,"15min":1.3979999999999999},"min":{"1min":0.55700000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.619,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0600000000000001,"15min":1.0840000000000001},"min":{"1min":0.78700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.423,"5min":2.5,"15min":2.5},"last":1.238}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.367},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.49}]},{"osd":7,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1279999999999999,"15min":1.1459999999999999},"min":{"1min":0.879,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.575,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.139,"15min":1.181},"min":{"1min":0.68700000000000006,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.534,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.5980000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:02:08.581 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:02:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:08.874 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:02:08.885 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-16T13:02:08.290504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":1005,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21120,"num_write_kb":309526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19275,"ondisk_log_size":19275,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424272,"kb_used_data":16316,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752550448,"statfs":{"total":773094113280,"available":770611658752,"internally_reserved":0,"allocated":16707584,"data_stored":9569383,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.475073"},"pg_stats":[{"pgid":"4.1f","version":"151'339","reported_seq":4229,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865786+0000","last_change":"2024-09-16T13:01:38.032360+0000","last_active":"2024-09-16T13:02:02.865786+0000","last_peered":"2024-09-16T13:02:02.865786+0000","last_clean":"2024-09-16T13:02:02.865786+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:02:02.865786+0000","last_undegraded":"2024-09-16T13:02:02.865786+0000","last_fullsized":"2024-09-16T13:02:02.865786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:11.756487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015622850000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":333,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"150'514","reported_seq":17234,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856527+0000","last_change":"2024-09-16T13:01:38.062519+0000","last_active":"2024-09-16T13:02:02.856527+0000","last_peered":"2024-09-16T13:02:02.856527+0000","last_clean":"2024-09-16T13:02:02.856527+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:02:02.856527+0000","last_undegraded":"2024-09-16T13:02:02.856527+0000","last_fullsized":"2024-09-16T13:02:02.856527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"142'433","reported_seq":4984,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829498+0000","last_change":"2024-09-16T13:01:38.020672+0000","last_active":"2024-09-16T13:02:02.829498+0000","last_peered":"2024-09-16T13:02:02.829498+0000","last_clean":"2024-09-16T13:02:02.829498+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:02:02.829498+0000","last_undegraded":"2024-09-16T13:02:02.829498+0000","last_fullsized":"2024-09-16T13:02:02.829498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"151'1171","reported_seq":8013,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863870+0000","last_change":"2024-09-16T13:01:38.017984+0000","last_active":"2024-09-16T13:02:02.863870+0000","last_peered":"2024-09-16T13:02:02.863870+0000","last_clean":"2024-09-16T13:02:02.863870+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:02:02.863870+0000","last_undegraded":"2024-09-16T13:02:02.863870+0000","last_fullsized":"2024-09-16T13:02:02.863870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"150'493","reported_seq":22038,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817747+0000","last_change":"2024-09-16T13:01:38.026576+0000","last_active":"2024-09-16T13:02:01.817747+0000","last_peered":"2024-09-16T13:02:01.817747+0000","last_clean":"2024-09-16T13:02:01.817747+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:02:01.817747+0000","last_undegraded":"2024-09-16T13:02:01.817747+0000","last_fullsized":"2024-09-16T13:02:01.817747+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:04.244300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"150'685","reported_seq":7141,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817710+0000","last_change":"2024-09-16T13:01:38.026337+0000","last_active":"2024-09-16T13:02:01.817710+0000","last_peered":"2024-09-16T13:02:01.817710+0000","last_clean":"2024-09-16T13:02:01.817710+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:02:01.817710+0000","last_undegraded":"2024-09-16T13:02:01.817710+0000","last_fullsized":"2024-09-16T13:02:01.817710+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:30.964357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"149'399","reported_seq":5474,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828856+0000","last_change":"2024-09-16T13:01:38.021608+0000","last_active":"2024-09-16T13:02:02.828856+0000","last_peered":"2024-09-16T13:02:02.828856+0000","last_clean":"2024-09-16T13:02:02.828856+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:02:02.828856+0000","last_undegraded":"2024-09-16T13:02:02.828856+0000","last_fullsized":"2024-09-16T13:02:02.828856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"150'440","reported_seq":6455,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817051+0000","last_change":"2024-09-16T13:01:38.024730+0000","last_active":"2024-09-16T13:02:01.817051+0000","last_peered":"2024-09-16T13:02:01.817051+0000","last_clean":"2024-09-16T13:02:01.817051+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:02:01.817051+0000","last_undegraded":"2024-09-16T13:02:01.817051+0000","last_fullsized":"2024-09-16T13:02:01.817051+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:23.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"150'889","reported_seq":7631,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829608+0000","last_change":"2024-09-16T13:01:38.021503+0000","last_active":"2024-09-16T13:02:02.829608+0000","last_peered":"2024-09-16T13:02:02.829608+0000","last_clean":"2024-09-16T13:02:02.829608+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:02:02.829608+0000","last_undegraded":"2024-09-16T13:02:02.829608+0000","last_fullsized":"2024-09-16T13:02:02.829608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"150'508","reported_seq":3310,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856528+0000","last_change":"2024-09-16T13:01:38.062149+0000","last_active":"2024-09-16T13:02:02.856528+0000","last_peered":"2024-09-16T13:02:02.856528+0000","last_clean":"2024-09-16T13:02:02.856528+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:02:02.856528+0000","last_undegraded":"2024-09-16T13:02:02.856528+0000","last_fullsized":"2024-09-16T13:02:02.856528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"150'523","reported_seq":16214,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829042+0000","last_change":"2024-09-16T13:01:38.021224+0000","last_active":"2024-09-16T13:02:02.829042+0000","last_peered":"2024-09-16T13:02:02.829042+0000","last_clean":"2024-09-16T13:02:02.829042+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:02:02.829042+0000","last_undegraded":"2024-09-16T13:02:02.829042+0000","last_fullsized":"2024-09-16T13:02:02.829042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"149'579","reported_seq":23227,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829478+0000","last_change":"2024-09-16T13:01:38.020686+0000","last_active":"2024-09-16T13:02:02.829478+0000","last_peered":"2024-09-16T13:02:02.829478+0000","last_clean":"2024-09-16T13:02:02.829478+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:02:02.829478+0000","last_undegraded":"2024-09-16T13:02:02.829478+0000","last_fullsized":"2024-09-16T13:02:02.829478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"151'512","reported_seq":6231,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817958+0000","last_change":"2024-09-16T13:01:38.027017+0000","last_active":"2024-09-16T13:02:01.817958+0000","last_peered":"2024-09-16T13:02:01.817958+0000","last_clean":"2024-09-16T13:02:01.817958+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:02:01.817958+0000","last_undegraded":"2024-09-16T13:02:01.817958+0000","last_fullsized":"2024-09-16T13:02:01.817958+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:56.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"150'371","reported_seq":3123,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828778+0000","last_change":"2024-09-16T13:01:38.019785+0000","last_active":"2024-09-16T13:02:02.828778+0000","last_peered":"2024-09-16T13:02:02.828778+0000","last_clean":"2024-09-16T13:02:02.828778+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:02:02.828778+0000","last_undegraded":"2024-09-16T13:02:02.828778+0000","last_fullsized":"2024-09-16T13:02:02.828778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"149'363","reported_seq":18827,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828605+0000","last_change":"2024-09-16T13:01:38.019836+0000","last_active":"2024-09-16T13:02:02.828605+0000","last_peered":"2024-09-16T13:02:02.828605+0000","last_clean":"2024-09-16T13:02:02.828605+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:02:02.828605+0000","last_undegraded":"2024-09-16T13:02:02.828605+0000","last_fullsized":"2024-09-16T13:02:02.828605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"139'335","reported_seq":5458,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.018514+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865667+0000","last_change":"2024-09-16T13:01:35.025514+0000","last_active":"2024-09-16T13:02:02.865667+0000","last_peered":"2024-09-16T13:02:02.865667+0000","last_clean":"2024-09-16T13:02:02.865667+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:02:02.865667+0000","last_undegraded":"2024-09-16T13:02:02.865667+0000","last_fullsized":"2024-09-16T13:02:02.865667+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:27.902219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"146'451","reported_seq":3774,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817602+0000","last_change":"2024-09-16T13:01:38.027021+0000","last_active":"2024-09-16T13:02:01.817602+0000","last_peered":"2024-09-16T13:02:01.817602+0000","last_clean":"2024-09-16T13:02:01.817602+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:02:01.817602+0000","last_undegraded":"2024-09-16T13:02:01.817602+0000","last_fullsized":"2024-09-16T13:02:01.817602+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460996+0000","last_change":"2024-09-16T13:01:34.980687+0000","last_active":"2024-09-16T13:02:02.460996+0000","last_peered":"2024-09-16T13:02:02.460996+0000","last_clean":"2024-09-16T13:02:02.460996+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:02:02.460996+0000","last_undegraded":"2024-09-16T13:02:02.460996+0000","last_fullsized":"2024-09-16T13:02:02.460996+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:44.616572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"149'690","reported_seq":7327,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.020063+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":160,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459793+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:02:02.459793+0000","last_peered":"2024-09-16T13:02:02.459793+0000","last_clean":"2024-09-16T13:02:02.459793+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:02:02.459793+0000","last_undegraded":"2024-09-16T13:02:02.459793+0000","last_fullsized":"2024-09-16T13:02:02.459793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"151'687","reported_seq":3219,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856470+0000","last_change":"2024-09-16T13:01:38.061542+0000","last_active":"2024-09-16T13:02:02.856470+0000","last_peered":"2024-09-16T13:02:02.856470+0000","last_clean":"2024-09-16T13:02:02.856470+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:02:02.856470+0000","last_undegraded":"2024-09-16T13:02:02.856470+0000","last_fullsized":"2024-09-16T13:02:02.856470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460629+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:02:02.460629+0000","last_peered":"2024-09-16T13:02:02.460629+0000","last_clean":"2024-09-16T13:02:02.460629+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:02:02.460629+0000","last_undegraded":"2024-09-16T13:02:02.460629+0000","last_fullsized":"2024-09-16T13:02:02.460629+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"150'586","reported_seq":17109,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460622+0000","last_change":"2024-09-16T13:01:38.114806+0000","last_active":"2024-09-16T13:02:02.460622+0000","last_peered":"2024-09-16T13:02:02.460622+0000","last_clean":"2024-09-16T13:02:02.460622+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:02:02.460622+0000","last_undegraded":"2024-09-16T13:02:02.460622+0000","last_fullsized":"2024-09-16T13:02:02.460622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":123,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817718+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:02:01.817718+0000","last_peered":"2024-09-16T13:02:01.817718+0000","last_clean":"2024-09-16T13:02:01.817718+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:02:01.817718+0000","last_undegraded":"2024-09-16T13:02:01.817718+0000","last_fullsized":"2024-09-16T13:02:01.817718+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":191,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829038+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:02:02.829038+0000","last_peered":"2024-09-16T13:02:02.829038+0000","last_clean":"2024-09-16T13:02:02.829038+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:02:02.829038+0000","last_undegraded":"2024-09-16T13:02:02.829038+0000","last_fullsized":"2024-09-16T13:02:02.829038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"148'501","reported_seq":17937,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865403+0000","last_change":"2024-09-16T13:01:38.031901+0000","last_active":"2024-09-16T13:02:02.865403+0000","last_peered":"2024-09-16T13:02:02.865403+0000","last_clean":"2024-09-16T13:02:02.865403+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:02:02.865403+0000","last_undegraded":"2024-09-16T13:02:02.865403+0000","last_fullsized":"2024-09-16T13:02:02.865403+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:27.630481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411569999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":121,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460348+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:02:02.460348+0000","last_peered":"2024-09-16T13:02:02.460348+0000","last_clean":"2024-09-16T13:02:02.460348+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:02:02.460348+0000","last_undegraded":"2024-09-16T13:02:02.460348+0000","last_fullsized":"2024-09-16T13:02:02.460348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863815+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:02:02.863815+0000","last_peered":"2024-09-16T13:02:02.863815+0000","last_clean":"2024-09-16T13:02:02.863815+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:02:02.863815+0000","last_undegraded":"2024-09-16T13:02:02.863815+0000","last_fullsized":"2024-09-16T13:02:02.863815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'407","reported_seq":2774,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.461059+0000","last_change":"2024-09-16T13:01:38.113780+0000","last_active":"2024-09-16T13:02:02.461059+0000","last_peered":"2024-09-16T13:02:02.461059+0000","last_clean":"2024-09-16T13:02:02.461059+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:02:02.461059+0000","last_undegraded":"2024-09-16T13:02:02.461059+0000","last_fullsized":"2024-09-16T13:02:02.461059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":169,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856355+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:02:02.856355+0000","last_peered":"2024-09-16T13:02:02.856355+0000","last_clean":"2024-09-16T13:02:02.856355+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:02:02.856355+0000","last_undegraded":"2024-09-16T13:02:02.856355+0000","last_fullsized":"2024-09-16T13:02:02.856355+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459690+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:02:02.459690+0000","last_peered":"2024-09-16T13:02:02.459690+0000","last_clean":"2024-09-16T13:02:02.459690+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:02:02.459690+0000","last_undegraded":"2024-09-16T13:02:02.459690+0000","last_fullsized":"2024-09-16T13:02:02.459690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"151'434","reported_seq":7322,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856609+0000","last_change":"2024-09-16T13:01:38.062717+0000","last_active":"2024-09-16T13:02:02.856609+0000","last_peered":"2024-09-16T13:02:02.856609+0000","last_clean":"2024-09-16T13:02:02.856609+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:02:02.856609+0000","last_undegraded":"2024-09-16T13:02:02.856609+0000","last_fullsized":"2024-09-16T13:02:02.856609+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"151'4","reported_seq":158,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829055+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:02:02.829055+0000","last_peered":"2024-09-16T13:02:02.829055+0000","last_clean":"2024-09-16T13:02:02.829055+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:02:02.829055+0000","last_undegraded":"2024-09-16T13:02:02.829055+0000","last_fullsized":"2024-09-16T13:02:02.829055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":164,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829054+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:02:02.829054+0000","last_peered":"2024-09-16T13:02:02.829054+0000","last_clean":"2024-09-16T13:02:02.829054+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:02:02.829054+0000","last_undegraded":"2024-09-16T13:02:02.829054+0000","last_fullsized":"2024-09-16T13:02:02.829054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":248,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459761+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:02:02.459761+0000","last_peered":"2024-09-16T13:02:02.459761+0000","last_clean":"2024-09-16T13:02:02.459761+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:02:02.459761+0000","last_undegraded":"2024-09-16T13:02:02.459761+0000","last_fullsized":"2024-09-16T13:02:02.459761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"150'444","reported_seq":3720,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865541+0000","last_change":"2024-09-16T13:01:38.031044+0000","last_active":"2024-09-16T13:02:02.865541+0000","last_peered":"2024-09-16T13:02:02.865541+0000","last_clean":"2024-09-16T13:02:02.865541+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:02:02.865541+0000","last_undegraded":"2024-09-16T13:02:02.865541+0000","last_fullsized":"2024-09-16T13:02:02.865541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:16.451880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602565,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"150'977","reported_seq":18782,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828748+0000","last_change":"2024-09-16T13:01:38.019797+0000","last_active":"2024-09-16T13:02:02.828748+0000","last_peered":"2024-09-16T13:02:02.828748+0000","last_clean":"2024-09-16T13:02:02.828748+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:02:02.828748+0000","last_undegraded":"2024-09-16T13:02:02.828748+0000","last_fullsized":"2024-09-16T13:02:02.828748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"149'328","reported_seq":846,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817233+0000","last_change":"2024-09-16T13:01:38.024649+0000","last_active":"2024-09-16T13:02:01.817233+0000","last_peered":"2024-09-16T13:02:01.817233+0000","last_clean":"2024-09-16T13:02:01.817233+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:02:01.817233+0000","last_undegraded":"2024-09-16T13:02:01.817233+0000","last_fullsized":"2024-09-16T13:02:01.817233+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"150'1380","reported_seq":6972,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865373+0000","last_change":"2024-09-16T13:01:38.031686+0000","last_active":"2024-09-16T13:02:02.865373+0000","last_peered":"2024-09-16T13:02:02.865373+0000","last_clean":"2024-09-16T13:02:02.865373+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:02:02.865373+0000","last_undegraded":"2024-09-16T13:02:02.865373+0000","last_fullsized":"2024-09-16T13:02:02.865373+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:00.052447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093887699999999996,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"144'974","reported_seq":6637,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856424+0000","last_change":"2024-09-16T13:01:38.060958+0000","last_active":"2024-09-16T13:02:02.856424+0000","last_peered":"2024-09-16T13:02:02.856424+0000","last_clean":"2024-09-16T13:02:02.856424+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:02:02.856424+0000","last_undegraded":"2024-09-16T13:02:02.856424+0000","last_fullsized":"2024-09-16T13:02:02.856424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"151'553","reported_seq":5349,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817849+0000","last_change":"2024-09-16T13:01:38.025883+0000","last_active":"2024-09-16T13:02:01.817849+0000","last_peered":"2024-09-16T13:02:01.817849+0000","last_clean":"2024-09-16T13:02:01.817849+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:02:01.817849+0000","last_undegraded":"2024-09-16T13:02:01.817849+0000","last_fullsized":"2024-09-16T13:02:01.817849+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"150'434","reported_seq":4404,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817961+0000","last_change":"2024-09-16T13:01:38.027791+0000","last_active":"2024-09-16T13:02:01.817961+0000","last_peered":"2024-09-16T13:02:01.817961+0000","last_clean":"2024-09-16T13:02:01.817961+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:02:01.817961+0000","last_undegraded":"2024-09-16T13:02:01.817961+0000","last_fullsized":"2024-09-16T13:02:01.817961+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"150'876","reported_seq":4050,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.818018+0000","last_change":"2024-09-16T13:01:38.025990+0000","last_active":"2024-09-16T13:02:01.818018+0000","last_peered":"2024-09-16T13:02:01.818018+0000","last_clean":"2024-09-16T13:02:01.818018+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:02:01.818018+0000","last_undegraded":"2024-09-16T13:02:01.818018+0000","last_fullsized":"2024-09-16T13:02:01.818018+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:12.558678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"149'919","reported_seq":36748,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.461057+0000","last_change":"2024-09-16T13:01:38.113695+0000","last_active":"2024-09-16T13:02:02.461057+0000","last_peered":"2024-09-16T13:02:02.461057+0000","last_clean":"2024-09-16T13:02:02.461057+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:02:02.461057+0000","last_undegraded":"2024-09-16T13:02:02.461057+0000","last_fullsized":"2024-09-16T13:02:02.461057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21015,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":19195,"ondisk_log_size":19195,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192521,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301756,"kb_used_data":2028,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070084,"statfs":{"total":96636764160,"available":96327766016,"internally_reserved":0,"allocated":2076672,"data_stored":1051715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]}]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303568,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068272,"statfs":{"total":96636764160,"available":96325910528,"internally_reserved":0,"allocated":2088960,"data_stored":1051902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.85599999999999998},"min":{"1min":0.48099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1499999999999999,"5min":1.272,"15min":1.272},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.83999999999999997},"min":{"1min":0.55100000000000005,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.391,"15min":1.391},"last":0.57899999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.97299999999999998,"15min":0.89200000000000002},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.333,"5min":14.275,"15min":14.275},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.58699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3600000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.752}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.58099999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2430000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.996,"5min":1.071,"15min":1.103},"min":{"1min":0.61799999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4830000000000001,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.95299999999999996}]},{"osd":3,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.071,"15min":1.0669999999999999},"min":{"1min":0.61899999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.2989999999999999,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.115},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6299999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.272}]},{"osd":4,"last update":"Mon Sep 16 13:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0109999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":1.446},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.077,"15min":1.137},"min":{"1min":0.65300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4379999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.29},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1850000000000001,"15min":1.2450000000000001},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.673,"5min":1.748,"15min":1.748},"last":1.369}]}]},{"osd":4,"up_from":108,"seq":463856467984,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309436,"kb_used_data":2020,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062404,"statfs":{"total":96636764160,"available":96319901696,"internally_reserved":0,"allocated":2068480,"data_stored":1045486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.0309999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.3899999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.23},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.4630000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.4339999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]},{"osd":7,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.498}]}]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299392,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072448,"statfs":{"total":96636764160,"available":96330186752,"internally_reserved":0,"allocated":2076672,"data_stored":1045654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.995,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71799999999999997,"15min":0.66700000000000004},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0940000000000001,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.232,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.873},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.51500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.103,"5min":1.528,"15min":1.528},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98399999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.49299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5740000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.0529999999999999}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.165}]},{"osd":5,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.109,"15min":1.129},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8520000000000001,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.171,"5min":1.1060000000000001,"15min":1.1180000000000001},"min":{"1min":0.56000000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.143}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1299999999999999},"min":{"1min":0.61099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.363},{"interface":"front","average":{"1min":1.167,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.77700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9830000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.304}]}]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299388,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072452,"statfs":{"total":96636764160,"available":96330190848,"internally_reserved":0,"allocated":2072576,"data_stored":1051222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.314,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.0980000000000001,"5min":10.654,"15min":10.654},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.77100000000000002},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.97899999999999998,"5min":10.816000000000001,"15min":10.816000000000001},"last":1.2589999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.52200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99099999999999999,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.97999999999999998}]},{"osd":3,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.89200000000000002,"15min":0.80400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.2430000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.077,"5min":1.647,"15min":1.647},"last":0.873}]},{"osd":4,"last update":"Mon Sep 16 13:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.365}]},{"osd":5,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.093,"15min":1.1930000000000001},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.226,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.516},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0740000000000001,"15min":1.1439999999999999},"min":{"1min":0.60799999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.383,"5min":2.125,"15min":2.125},"last":1.772}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.8089999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.681}]},{"osd":7,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.139,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.478,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.7350000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.1539999999999999,"15min":1.2450000000000001},"min":{"1min":0.63700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.538,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.901}]}]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303600,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068240,"statfs":{"total":96636764160,"available":96325877760,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.88600000000000001,"15min":0.84199999999999997},"min":{"1min":0.39100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.1899999999999999,"5min":9.407,"15min":9.407},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.85999999999999999,"15min":0.94599999999999995},"min":{"1min":0.46899999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.319,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0469999999999999,"5min":2.254,"15min":2.254},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.36399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83199999999999996,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":1.028},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4750000000000001,"5min":2.044,"15min":2.044},"last":0.879}]},{"osd":4,"last update":"Mon Sep 16 13:01:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.038},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.177}]},{"osd":5,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0149999999999999,"15min":1.0640000000000001},"min":{"1min":0.56399999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4730000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.999,"5min":1.133,"15min":1.2050000000000001},"min":{"1min":0.60999999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.325,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.125}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.145}]},{"osd":7,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.129,"15min":1.177},"min":{"1min":0.745,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.327,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.093},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1599999999999999},"min":{"1min":0.70099999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.427,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.2010000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303596,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068244,"statfs":{"total":96636764160,"available":96325881856,"internally_reserved":0,"allocated":2121728,"data_stored":1635835,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5920000000000001,"5min":1.73,"15min":1.73},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.145,"15min":1.26},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3080000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.98199999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0249999999999999,"15min":1.1559999999999999},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.208,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.56499999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5149999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.81999999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0569999999999999},"min":{"1min":0.55400000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.298,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.143,"15min":1.2030000000000001},"min":{"1min":0.50900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1839999999999999,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.0189999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0109999999999999,"15min":1.0720000000000001},"min":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.984,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.117},{"interface":"front","average":{"1min":1.028,"5min":1.2010000000000001,"15min":1.298},"min":{"1min":0.66900000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9299999999999999,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.272}]},{"osd":4,"last update":"Mon Sep 16 13:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.046},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.96199999999999997}]},{"osd":5,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.1759999999999999,"15min":1.3029999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.135,"5min":6.282,"15min":6.282},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.93600000000000005,"15min":0.97799999999999998},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.2549999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.153}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.083},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.349}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303536,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068304,"statfs":{"total":96636764160,"available":96325943296,"internally_reserved":0,"allocated":2068480,"data_stored":1045486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.73999999999999999},"min":{"1min":0.38100000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.97799999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.254,"15min":0.254},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72799999999999998,"15min":0.77600000000000002},"min":{"1min":0.35699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.90400000000000003,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.181,"5min":3.024,"15min":3.024},"last":1.1990000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.95999999999999996,"15min":1.034},"min":{"1min":0.53600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2250000000000001,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.49199999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.151,"5min":3.544,"15min":3.544},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.034},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.25,"15min":1.3979999999999999},"min":{"1min":0.55700000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.619,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0600000000000001,"15min":1.0840000000000001},"min":{"1min":0.78700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.423,"5min":2.5,"15min":2.5},"last":1.238}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.367},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.49}]},{"osd":7,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1279999999999999,"15min":1.1459999999999999},"min":{"1min":0.879,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.575,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.139,"15min":1.181},"min":{"1min":0.68700000000000006,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.534,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.5980000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:02:08.888 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T13:02:08.889 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-16T13:02:09.184 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T13:02:09.193 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-16T13:02:09.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T13:02:09.495 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:09.496 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":173,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T13:02:03.831869+0000","last_up_change":"2024-09-16T13:01:33.961419+0000","last_in_change":"2024-09-16T12:59:39.968761+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T12:58:44.284177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T12:58:51.733915+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T12:59:00.193651+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T12:59:09.779825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":79,"snap_epoch":151,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":22186},{"type":"v1","addr":"172.21.15.42:6819","nonce":22186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":22186},{"type":"v1","addr":"172.21.15.42:6823","nonce":22186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":22186},{"type":"v1","addr":"172.21.15.42:6830","nonce":22186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":22186},{"type":"v1","addr":"172.21.15.42:6826","nonce":22186}]},"public_addr":"172.21.15.42:6819/22186","cluster_addr":"172.21.15.42:6823/22186","heartbeat_back_addr":"172.21.15.42:6830/22186","heartbeat_front_addr":"172.21.15.42:6826/22186","state":["exists","up"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":22184},{"type":"v1","addr":"172.21.15.42:6809","nonce":22184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":22184},{"type":"v1","addr":"172.21.15.42:6811","nonce":22184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":22184},{"type":"v1","addr":"172.21.15.42:6815","nonce":22184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":22184},{"type":"v1","addr":"172.21.15.42:6813","nonce":22184}]},"public_addr":"172.21.15.42:6809/22184","cluster_addr":"172.21.15.42:6811/22184","heartbeat_back_addr":"172.21.15.42:6815/22184","heartbeat_front_addr":"172.21.15.42:6813/22184","state":["exists","up"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":22185},{"type":"v1","addr":"172.21.15.42:6801","nonce":22185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":22185},{"type":"v1","addr":"172.21.15.42:6803","nonce":22185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":22185},{"type":"v1","addr":"172.21.15.42:6807","nonce":22185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":22185},{"type":"v1","addr":"172.21.15.42:6805","nonce":22185}]},"public_addr":"172.21.15.42:6801/22185","cluster_addr":"172.21.15.42:6803/22185","heartbeat_back_addr":"172.21.15.42:6807/22185","heartbeat_front_addr":"172.21.15.42:6805/22185","state":["exists","up"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":22187},{"type":"v1","addr":"172.21.15.42:6818","nonce":22187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":22187},{"type":"v1","addr":"172.21.15.42:6822","nonce":22187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":22187},{"type":"v1","addr":"172.21.15.42:6831","nonce":22187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":22187},{"type":"v1","addr":"172.21.15.42:6827","nonce":22187}]},"public_addr":"172.21.15.42:6818/22187","cluster_addr":"172.21.15.42:6822/22187","heartbeat_back_addr":"172.21.15.42:6831/22187","heartbeat_front_addr":"172.21.15.42:6827/22187","state":["exists","up"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":83,"up_from":108,"up_thru":147,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":33598},{"type":"v1","addr":"172.21.15.136:6817","nonce":33598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":33598},{"type":"v1","addr":"172.21.15.136:6819","nonce":33598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":33598},{"type":"v1","addr":"172.21.15.136:6823","nonce":33598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":33598},{"type":"v1","addr":"172.21.15.136:6821","nonce":33598}]},"public_addr":"172.21.15.136:6817/33598","cluster_addr":"172.21.15.136:6819/33598","heartbeat_back_addr":"172.21.15.136:6823/33598","heartbeat_front_addr":"172.21.15.136:6821/33598","state":["exists","up"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22340},{"type":"v1","addr":"172.21.15.136:6809","nonce":22340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22340},{"type":"v1","addr":"172.21.15.136:6811","nonce":22340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22340},{"type":"v1","addr":"172.21.15.136:6815","nonce":22340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22340},{"type":"v1","addr":"172.21.15.136:6813","nonce":22340}]},"public_addr":"172.21.15.136:6809/22340","cluster_addr":"172.21.15.136:6811/22340","heartbeat_back_addr":"172.21.15.136:6815/22340","heartbeat_front_addr":"172.21.15.136:6813/22340","state":["exists","up"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":120,"up_from":147,"up_thru":147,"down_at":121,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":41077},{"type":"v1","addr":"172.21.15.136:6825","nonce":41077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":41077},{"type":"v1","addr":"172.21.15.136:6827","nonce":41077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":41077},{"type":"v1","addr":"172.21.15.136:6831","nonce":41077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":41077},{"type":"v1","addr":"172.21.15.136:6829","nonce":41077}]},"public_addr":"172.21.15.136:6825/41077","cluster_addr":"172.21.15.136:6827/41077","heartbeat_back_addr":"172.21.15.136:6831/41077","heartbeat_front_addr":"172.21.15.136:6829/41077","state":["exists","up"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22341},{"type":"v1","addr":"172.21.15.136:6801","nonce":22341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22341},{"type":"v1","addr":"172.21.15.136:6803","nonce":22341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22341},{"type":"v1","addr":"172.21.15.136:6807","nonce":22341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22341},{"type":"v1","addr":"172.21.15.136:6805","nonce":22341}]},"public_addr":"172.21.15.136:6801/22341","cluster_addr":"172.21.15.136:6803/22341","heartbeat_back_addr":"172.21.15.136:6807/22341","heartbeat_front_addr":"172.21.15.136:6805/22341","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.264463+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.214474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.217927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.435822+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.451286+0000","dead_epoch":84},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.287403+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.440296+0000","dead_epoch":121},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.140630+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"4.12","osds":[5,1]}],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":1,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T13:02:09.508 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T13:02:09.508 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:02:09.812 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:09.814 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:02:09.827 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-16T13:02:08.290504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":1005,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21120,"num_write_kb":309526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19275,"ondisk_log_size":19275,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424272,"kb_used_data":16316,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752550448,"statfs":{"total":773094113280,"available":770611658752,"internally_reserved":0,"allocated":16707584,"data_stored":9569383,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.475073"},"pg_stats":[{"pgid":"4.1f","version":"151'339","reported_seq":4229,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865786+0000","last_change":"2024-09-16T13:01:38.032360+0000","last_active":"2024-09-16T13:02:02.865786+0000","last_peered":"2024-09-16T13:02:02.865786+0000","last_clean":"2024-09-16T13:02:02.865786+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:02:02.865786+0000","last_undegraded":"2024-09-16T13:02:02.865786+0000","last_fullsized":"2024-09-16T13:02:02.865786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:11.756487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015622850000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":333,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"150'514","reported_seq":17234,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856527+0000","last_change":"2024-09-16T13:01:38.062519+0000","last_active":"2024-09-16T13:02:02.856527+0000","last_peered":"2024-09-16T13:02:02.856527+0000","last_clean":"2024-09-16T13:02:02.856527+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:02:02.856527+0000","last_undegraded":"2024-09-16T13:02:02.856527+0000","last_fullsized":"2024-09-16T13:02:02.856527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"142'433","reported_seq":4984,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829498+0000","last_change":"2024-09-16T13:01:38.020672+0000","last_active":"2024-09-16T13:02:02.829498+0000","last_peered":"2024-09-16T13:02:02.829498+0000","last_clean":"2024-09-16T13:02:02.829498+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:02:02.829498+0000","last_undegraded":"2024-09-16T13:02:02.829498+0000","last_fullsized":"2024-09-16T13:02:02.829498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"151'1171","reported_seq":8013,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863870+0000","last_change":"2024-09-16T13:01:38.017984+0000","last_active":"2024-09-16T13:02:02.863870+0000","last_peered":"2024-09-16T13:02:02.863870+0000","last_clean":"2024-09-16T13:02:02.863870+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:02:02.863870+0000","last_undegraded":"2024-09-16T13:02:02.863870+0000","last_fullsized":"2024-09-16T13:02:02.863870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"150'493","reported_seq":22038,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817747+0000","last_change":"2024-09-16T13:01:38.026576+0000","last_active":"2024-09-16T13:02:01.817747+0000","last_peered":"2024-09-16T13:02:01.817747+0000","last_clean":"2024-09-16T13:02:01.817747+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:02:01.817747+0000","last_undegraded":"2024-09-16T13:02:01.817747+0000","last_fullsized":"2024-09-16T13:02:01.817747+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:04.244300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"150'685","reported_seq":7141,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817710+0000","last_change":"2024-09-16T13:01:38.026337+0000","last_active":"2024-09-16T13:02:01.817710+0000","last_peered":"2024-09-16T13:02:01.817710+0000","last_clean":"2024-09-16T13:02:01.817710+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:02:01.817710+0000","last_undegraded":"2024-09-16T13:02:01.817710+0000","last_fullsized":"2024-09-16T13:02:01.817710+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:30.964357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"149'399","reported_seq":5474,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828856+0000","last_change":"2024-09-16T13:01:38.021608+0000","last_active":"2024-09-16T13:02:02.828856+0000","last_peered":"2024-09-16T13:02:02.828856+0000","last_clean":"2024-09-16T13:02:02.828856+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:02:02.828856+0000","last_undegraded":"2024-09-16T13:02:02.828856+0000","last_fullsized":"2024-09-16T13:02:02.828856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"150'440","reported_seq":6455,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817051+0000","last_change":"2024-09-16T13:01:38.024730+0000","last_active":"2024-09-16T13:02:01.817051+0000","last_peered":"2024-09-16T13:02:01.817051+0000","last_clean":"2024-09-16T13:02:01.817051+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:02:01.817051+0000","last_undegraded":"2024-09-16T13:02:01.817051+0000","last_fullsized":"2024-09-16T13:02:01.817051+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:23.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"150'889","reported_seq":7631,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829608+0000","last_change":"2024-09-16T13:01:38.021503+0000","last_active":"2024-09-16T13:02:02.829608+0000","last_peered":"2024-09-16T13:02:02.829608+0000","last_clean":"2024-09-16T13:02:02.829608+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:02:02.829608+0000","last_undegraded":"2024-09-16T13:02:02.829608+0000","last_fullsized":"2024-09-16T13:02:02.829608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"150'508","reported_seq":3310,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856528+0000","last_change":"2024-09-16T13:01:38.062149+0000","last_active":"2024-09-16T13:02:02.856528+0000","last_peered":"2024-09-16T13:02:02.856528+0000","last_clean":"2024-09-16T13:02:02.856528+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:02:02.856528+0000","last_undegraded":"2024-09-16T13:02:02.856528+0000","last_fullsized":"2024-09-16T13:02:02.856528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"150'523","reported_seq":16214,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829042+0000","last_change":"2024-09-16T13:01:38.021224+0000","last_active":"2024-09-16T13:02:02.829042+0000","last_peered":"2024-09-16T13:02:02.829042+0000","last_clean":"2024-09-16T13:02:02.829042+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:02:02.829042+0000","last_undegraded":"2024-09-16T13:02:02.829042+0000","last_fullsized":"2024-09-16T13:02:02.829042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"149'579","reported_seq":23227,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829478+0000","last_change":"2024-09-16T13:01:38.020686+0000","last_active":"2024-09-16T13:02:02.829478+0000","last_peered":"2024-09-16T13:02:02.829478+0000","last_clean":"2024-09-16T13:02:02.829478+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:02:02.829478+0000","last_undegraded":"2024-09-16T13:02:02.829478+0000","last_fullsized":"2024-09-16T13:02:02.829478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"151'512","reported_seq":6231,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817958+0000","last_change":"2024-09-16T13:01:38.027017+0000","last_active":"2024-09-16T13:02:01.817958+0000","last_peered":"2024-09-16T13:02:01.817958+0000","last_clean":"2024-09-16T13:02:01.817958+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:02:01.817958+0000","last_undegraded":"2024-09-16T13:02:01.817958+0000","last_fullsized":"2024-09-16T13:02:01.817958+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:56.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"150'371","reported_seq":3123,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828778+0000","last_change":"2024-09-16T13:01:38.019785+0000","last_active":"2024-09-16T13:02:02.828778+0000","last_peered":"2024-09-16T13:02:02.828778+0000","last_clean":"2024-09-16T13:02:02.828778+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:02:02.828778+0000","last_undegraded":"2024-09-16T13:02:02.828778+0000","last_fullsized":"2024-09-16T13:02:02.828778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"149'363","reported_seq":18827,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828605+0000","last_change":"2024-09-16T13:01:38.019836+0000","last_active":"2024-09-16T13:02:02.828605+0000","last_peered":"2024-09-16T13:02:02.828605+0000","last_clean":"2024-09-16T13:02:02.828605+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:02:02.828605+0000","last_undegraded":"2024-09-16T13:02:02.828605+0000","last_fullsized":"2024-09-16T13:02:02.828605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"139'335","reported_seq":5458,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.018514+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865667+0000","last_change":"2024-09-16T13:01:35.025514+0000","last_active":"2024-09-16T13:02:02.865667+0000","last_peered":"2024-09-16T13:02:02.865667+0000","last_clean":"2024-09-16T13:02:02.865667+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:02:02.865667+0000","last_undegraded":"2024-09-16T13:02:02.865667+0000","last_fullsized":"2024-09-16T13:02:02.865667+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:27.902219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"146'451","reported_seq":3774,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817602+0000","last_change":"2024-09-16T13:01:38.027021+0000","last_active":"2024-09-16T13:02:01.817602+0000","last_peered":"2024-09-16T13:02:01.817602+0000","last_clean":"2024-09-16T13:02:01.817602+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:02:01.817602+0000","last_undegraded":"2024-09-16T13:02:01.817602+0000","last_fullsized":"2024-09-16T13:02:01.817602+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460996+0000","last_change":"2024-09-16T13:01:34.980687+0000","last_active":"2024-09-16T13:02:02.460996+0000","last_peered":"2024-09-16T13:02:02.460996+0000","last_clean":"2024-09-16T13:02:02.460996+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:02:02.460996+0000","last_undegraded":"2024-09-16T13:02:02.460996+0000","last_fullsized":"2024-09-16T13:02:02.460996+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:44.616572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"149'690","reported_seq":7327,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.020063+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":160,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459793+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:02:02.459793+0000","last_peered":"2024-09-16T13:02:02.459793+0000","last_clean":"2024-09-16T13:02:02.459793+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:02:02.459793+0000","last_undegraded":"2024-09-16T13:02:02.459793+0000","last_fullsized":"2024-09-16T13:02:02.459793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"151'687","reported_seq":3219,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856470+0000","last_change":"2024-09-16T13:01:38.061542+0000","last_active":"2024-09-16T13:02:02.856470+0000","last_peered":"2024-09-16T13:02:02.856470+0000","last_clean":"2024-09-16T13:02:02.856470+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:02:02.856470+0000","last_undegraded":"2024-09-16T13:02:02.856470+0000","last_fullsized":"2024-09-16T13:02:02.856470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460629+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:02:02.460629+0000","last_peered":"2024-09-16T13:02:02.460629+0000","last_clean":"2024-09-16T13:02:02.460629+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:02:02.460629+0000","last_undegraded":"2024-09-16T13:02:02.460629+0000","last_fullsized":"2024-09-16T13:02:02.460629+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"150'586","reported_seq":17109,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460622+0000","last_change":"2024-09-16T13:01:38.114806+0000","last_active":"2024-09-16T13:02:02.460622+0000","last_peered":"2024-09-16T13:02:02.460622+0000","last_clean":"2024-09-16T13:02:02.460622+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:02:02.460622+0000","last_undegraded":"2024-09-16T13:02:02.460622+0000","last_fullsized":"2024-09-16T13:02:02.460622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":123,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817718+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:02:01.817718+0000","last_peered":"2024-09-16T13:02:01.817718+0000","last_clean":"2024-09-16T13:02:01.817718+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:02:01.817718+0000","last_undegraded":"2024-09-16T13:02:01.817718+0000","last_fullsized":"2024-09-16T13:02:01.817718+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":191,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829038+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:02:02.829038+0000","last_peered":"2024-09-16T13:02:02.829038+0000","last_clean":"2024-09-16T13:02:02.829038+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:02:02.829038+0000","last_undegraded":"2024-09-16T13:02:02.829038+0000","last_fullsized":"2024-09-16T13:02:02.829038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"148'501","reported_seq":17937,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865403+0000","last_change":"2024-09-16T13:01:38.031901+0000","last_active":"2024-09-16T13:02:02.865403+0000","last_peered":"2024-09-16T13:02:02.865403+0000","last_clean":"2024-09-16T13:02:02.865403+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:02:02.865403+0000","last_undegraded":"2024-09-16T13:02:02.865403+0000","last_fullsized":"2024-09-16T13:02:02.865403+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:27.630481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411569999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":121,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460348+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:02:02.460348+0000","last_peered":"2024-09-16T13:02:02.460348+0000","last_clean":"2024-09-16T13:02:02.460348+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:02:02.460348+0000","last_undegraded":"2024-09-16T13:02:02.460348+0000","last_fullsized":"2024-09-16T13:02:02.460348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863815+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:02:02.863815+0000","last_peered":"2024-09-16T13:02:02.863815+0000","last_clean":"2024-09-16T13:02:02.863815+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:02:02.863815+0000","last_undegraded":"2024-09-16T13:02:02.863815+0000","last_fullsized":"2024-09-16T13:02:02.863815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'407","reported_seq":2774,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.461059+0000","last_change":"2024-09-16T13:01:38.113780+0000","last_active":"2024-09-16T13:02:02.461059+0000","last_peered":"2024-09-16T13:02:02.461059+0000","last_clean":"2024-09-16T13:02:02.461059+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:02:02.461059+0000","last_undegraded":"2024-09-16T13:02:02.461059+0000","last_fullsized":"2024-09-16T13:02:02.461059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":169,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856355+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:02:02.856355+0000","last_peered":"2024-09-16T13:02:02.856355+0000","last_clean":"2024-09-16T13:02:02.856355+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:02:02.856355+0000","last_undegraded":"2024-09-16T13:02:02.856355+0000","last_fullsized":"2024-09-16T13:02:02.856355+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459690+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:02:02.459690+0000","last_peered":"2024-09-16T13:02:02.459690+0000","last_clean":"2024-09-16T13:02:02.459690+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:02:02.459690+0000","last_undegraded":"2024-09-16T13:02:02.459690+0000","last_fullsized":"2024-09-16T13:02:02.459690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"151'434","reported_seq":7322,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856609+0000","last_change":"2024-09-16T13:01:38.062717+0000","last_active":"2024-09-16T13:02:02.856609+0000","last_peered":"2024-09-16T13:02:02.856609+0000","last_clean":"2024-09-16T13:02:02.856609+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:02:02.856609+0000","last_undegraded":"2024-09-16T13:02:02.856609+0000","last_fullsized":"2024-09-16T13:02:02.856609+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"151'4","reported_seq":158,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829055+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:02:02.829055+0000","last_peered":"2024-09-16T13:02:02.829055+0000","last_clean":"2024-09-16T13:02:02.829055+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:02:02.829055+0000","last_undegraded":"2024-09-16T13:02:02.829055+0000","last_fullsized":"2024-09-16T13:02:02.829055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":164,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829054+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:02:02.829054+0000","last_peered":"2024-09-16T13:02:02.829054+0000","last_clean":"2024-09-16T13:02:02.829054+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:02:02.829054+0000","last_undegraded":"2024-09-16T13:02:02.829054+0000","last_fullsized":"2024-09-16T13:02:02.829054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":248,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459761+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:02:02.459761+0000","last_peered":"2024-09-16T13:02:02.459761+0000","last_clean":"2024-09-16T13:02:02.459761+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:02:02.459761+0000","last_undegraded":"2024-09-16T13:02:02.459761+0000","last_fullsized":"2024-09-16T13:02:02.459761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"150'444","reported_seq":3720,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865541+0000","last_change":"2024-09-16T13:01:38.031044+0000","last_active":"2024-09-16T13:02:02.865541+0000","last_peered":"2024-09-16T13:02:02.865541+0000","last_clean":"2024-09-16T13:02:02.865541+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:02:02.865541+0000","last_undegraded":"2024-09-16T13:02:02.865541+0000","last_fullsized":"2024-09-16T13:02:02.865541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:16.451880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602565,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"150'977","reported_seq":18782,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828748+0000","last_change":"2024-09-16T13:01:38.019797+0000","last_active":"2024-09-16T13:02:02.828748+0000","last_peered":"2024-09-16T13:02:02.828748+0000","last_clean":"2024-09-16T13:02:02.828748+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:02:02.828748+0000","last_undegraded":"2024-09-16T13:02:02.828748+0000","last_fullsized":"2024-09-16T13:02:02.828748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"149'328","reported_seq":846,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817233+0000","last_change":"2024-09-16T13:01:38.024649+0000","last_active":"2024-09-16T13:02:01.817233+0000","last_peered":"2024-09-16T13:02:01.817233+0000","last_clean":"2024-09-16T13:02:01.817233+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:02:01.817233+0000","last_undegraded":"2024-09-16T13:02:01.817233+0000","last_fullsized":"2024-09-16T13:02:01.817233+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"150'1380","reported_seq":6972,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865373+0000","last_change":"2024-09-16T13:01:38.031686+0000","last_active":"2024-09-16T13:02:02.865373+0000","last_peered":"2024-09-16T13:02:02.865373+0000","last_clean":"2024-09-16T13:02:02.865373+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:02:02.865373+0000","last_undegraded":"2024-09-16T13:02:02.865373+0000","last_fullsized":"2024-09-16T13:02:02.865373+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:00.052447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093887699999999996,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"144'974","reported_seq":6637,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856424+0000","last_change":"2024-09-16T13:01:38.060958+0000","last_active":"2024-09-16T13:02:02.856424+0000","last_peered":"2024-09-16T13:02:02.856424+0000","last_clean":"2024-09-16T13:02:02.856424+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:02:02.856424+0000","last_undegraded":"2024-09-16T13:02:02.856424+0000","last_fullsized":"2024-09-16T13:02:02.856424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"151'553","reported_seq":5349,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817849+0000","last_change":"2024-09-16T13:01:38.025883+0000","last_active":"2024-09-16T13:02:01.817849+0000","last_peered":"2024-09-16T13:02:01.817849+0000","last_clean":"2024-09-16T13:02:01.817849+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:02:01.817849+0000","last_undegraded":"2024-09-16T13:02:01.817849+0000","last_fullsized":"2024-09-16T13:02:01.817849+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"150'434","reported_seq":4404,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817961+0000","last_change":"2024-09-16T13:01:38.027791+0000","last_active":"2024-09-16T13:02:01.817961+0000","last_peered":"2024-09-16T13:02:01.817961+0000","last_clean":"2024-09-16T13:02:01.817961+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:02:01.817961+0000","last_undegraded":"2024-09-16T13:02:01.817961+0000","last_fullsized":"2024-09-16T13:02:01.817961+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"150'876","reported_seq":4050,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.818018+0000","last_change":"2024-09-16T13:01:38.025990+0000","last_active":"2024-09-16T13:02:01.818018+0000","last_peered":"2024-09-16T13:02:01.818018+0000","last_clean":"2024-09-16T13:02:01.818018+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:02:01.818018+0000","last_undegraded":"2024-09-16T13:02:01.818018+0000","last_fullsized":"2024-09-16T13:02:01.818018+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:12.558678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"149'919","reported_seq":36748,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.461057+0000","last_change":"2024-09-16T13:01:38.113695+0000","last_active":"2024-09-16T13:02:02.461057+0000","last_peered":"2024-09-16T13:02:02.461057+0000","last_clean":"2024-09-16T13:02:02.461057+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:02:02.461057+0000","last_undegraded":"2024-09-16T13:02:02.461057+0000","last_fullsized":"2024-09-16T13:02:02.461057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21015,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":19195,"ondisk_log_size":19195,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192521,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301756,"kb_used_data":2028,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070084,"statfs":{"total":96636764160,"available":96327766016,"internally_reserved":0,"allocated":2076672,"data_stored":1051715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]}]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303568,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068272,"statfs":{"total":96636764160,"available":96325910528,"internally_reserved":0,"allocated":2088960,"data_stored":1051902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.85599999999999998},"min":{"1min":0.48099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1499999999999999,"5min":1.272,"15min":1.272},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.83999999999999997},"min":{"1min":0.55100000000000005,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.391,"15min":1.391},"last":0.57899999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.97299999999999998,"15min":0.89200000000000002},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.333,"5min":14.275,"15min":14.275},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.58699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3600000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.752}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.58099999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2430000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.996,"5min":1.071,"15min":1.103},"min":{"1min":0.61799999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4830000000000001,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.95299999999999996}]},{"osd":3,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.071,"15min":1.0669999999999999},"min":{"1min":0.61899999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.2989999999999999,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.115},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6299999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.272}]},{"osd":4,"last update":"Mon Sep 16 13:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0109999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":1.446},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.077,"15min":1.137},"min":{"1min":0.65300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4379999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.29},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1850000000000001,"15min":1.2450000000000001},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.673,"5min":1.748,"15min":1.748},"last":1.369}]}]},{"osd":4,"up_from":108,"seq":463856467984,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309436,"kb_used_data":2020,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062404,"statfs":{"total":96636764160,"available":96319901696,"internally_reserved":0,"allocated":2068480,"data_stored":1045486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.0309999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.3899999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.23},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.4630000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.4339999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]},{"osd":7,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.498}]}]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299392,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072448,"statfs":{"total":96636764160,"available":96330186752,"internally_reserved":0,"allocated":2076672,"data_stored":1045654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.995,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71799999999999997,"15min":0.66700000000000004},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0940000000000001,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.232,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.873},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.51500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.103,"5min":1.528,"15min":1.528},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98399999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.49299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5740000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.0529999999999999}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.165}]},{"osd":5,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.109,"15min":1.129},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8520000000000001,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.171,"5min":1.1060000000000001,"15min":1.1180000000000001},"min":{"1min":0.56000000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.143}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1299999999999999},"min":{"1min":0.61099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.363},{"interface":"front","average":{"1min":1.167,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.77700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9830000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.304}]}]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299388,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072452,"statfs":{"total":96636764160,"available":96330190848,"internally_reserved":0,"allocated":2072576,"data_stored":1051222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.314,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.0980000000000001,"5min":10.654,"15min":10.654},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.77100000000000002},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.97899999999999998,"5min":10.816000000000001,"15min":10.816000000000001},"last":1.2589999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.52200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99099999999999999,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.97999999999999998}]},{"osd":3,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.89200000000000002,"15min":0.80400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.2430000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.077,"5min":1.647,"15min":1.647},"last":0.873}]},{"osd":4,"last update":"Mon Sep 16 13:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.365}]},{"osd":5,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.093,"15min":1.1930000000000001},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.226,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.516},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0740000000000001,"15min":1.1439999999999999},"min":{"1min":0.60799999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.383,"5min":2.125,"15min":2.125},"last":1.772}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.8089999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.681}]},{"osd":7,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.139,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.478,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.7350000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.1539999999999999,"15min":1.2450000000000001},"min":{"1min":0.63700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.538,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.901}]}]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303600,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068240,"statfs":{"total":96636764160,"available":96325877760,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.88600000000000001,"15min":0.84199999999999997},"min":{"1min":0.39100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.1899999999999999,"5min":9.407,"15min":9.407},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.85999999999999999,"15min":0.94599999999999995},"min":{"1min":0.46899999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.319,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0469999999999999,"5min":2.254,"15min":2.254},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.36399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83199999999999996,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":1.028},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4750000000000001,"5min":2.044,"15min":2.044},"last":0.879}]},{"osd":4,"last update":"Mon Sep 16 13:01:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.038},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.177}]},{"osd":5,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0149999999999999,"15min":1.0640000000000001},"min":{"1min":0.56399999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4730000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.999,"5min":1.133,"15min":1.2050000000000001},"min":{"1min":0.60999999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.325,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.125}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.145}]},{"osd":7,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.129,"15min":1.177},"min":{"1min":0.745,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.327,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.093},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1599999999999999},"min":{"1min":0.70099999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.427,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.2010000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303596,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068244,"statfs":{"total":96636764160,"available":96325881856,"internally_reserved":0,"allocated":2121728,"data_stored":1635835,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5920000000000001,"5min":1.73,"15min":1.73},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.145,"15min":1.26},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3080000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.98199999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0249999999999999,"15min":1.1559999999999999},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.208,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.56499999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5149999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.81999999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0569999999999999},"min":{"1min":0.55400000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.298,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.143,"15min":1.2030000000000001},"min":{"1min":0.50900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1839999999999999,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.0189999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0109999999999999,"15min":1.0720000000000001},"min":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.984,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.117},{"interface":"front","average":{"1min":1.028,"5min":1.2010000000000001,"15min":1.298},"min":{"1min":0.66900000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9299999999999999,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.272}]},{"osd":4,"last update":"Mon Sep 16 13:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.046},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.96199999999999997}]},{"osd":5,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.1759999999999999,"15min":1.3029999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.135,"5min":6.282,"15min":6.282},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.93600000000000005,"15min":0.97799999999999998},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.2549999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.153}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.083},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.349}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303536,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068304,"statfs":{"total":96636764160,"available":96325943296,"internally_reserved":0,"allocated":2068480,"data_stored":1045486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.73999999999999999},"min":{"1min":0.38100000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.97799999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.254,"15min":0.254},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72799999999999998,"15min":0.77600000000000002},"min":{"1min":0.35699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.90400000000000003,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.181,"5min":3.024,"15min":3.024},"last":1.1990000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.95999999999999996,"15min":1.034},"min":{"1min":0.53600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2250000000000001,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.49199999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.151,"5min":3.544,"15min":3.544},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.034},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.25,"15min":1.3979999999999999},"min":{"1min":0.55700000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.619,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0600000000000001,"15min":1.0840000000000001},"min":{"1min":0.78700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.423,"5min":2.5,"15min":2.5},"last":1.238}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.367},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.49}]},{"osd":7,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1279999999999999,"15min":1.1459999999999999},"min":{"1min":0.879,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.575,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.139,"15min":1.181},"min":{"1min":0.68700000000000006,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.534,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.5980000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:02:09.830 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:02:10.135 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:10.138 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:02:10.150 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-16T13:02:08.290504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":1005,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21120,"num_write_kb":309526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19275,"ondisk_log_size":19275,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424272,"kb_used_data":16316,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752550448,"statfs":{"total":773094113280,"available":770611658752,"internally_reserved":0,"allocated":16707584,"data_stored":9569383,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.475073"},"pg_stats":[{"pgid":"4.1f","version":"151'339","reported_seq":4229,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865786+0000","last_change":"2024-09-16T13:01:38.032360+0000","last_active":"2024-09-16T13:02:02.865786+0000","last_peered":"2024-09-16T13:02:02.865786+0000","last_clean":"2024-09-16T13:02:02.865786+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:02:02.865786+0000","last_undegraded":"2024-09-16T13:02:02.865786+0000","last_fullsized":"2024-09-16T13:02:02.865786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:11.756487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015622850000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":333,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"150'514","reported_seq":17234,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856527+0000","last_change":"2024-09-16T13:01:38.062519+0000","last_active":"2024-09-16T13:02:02.856527+0000","last_peered":"2024-09-16T13:02:02.856527+0000","last_clean":"2024-09-16T13:02:02.856527+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:02:02.856527+0000","last_undegraded":"2024-09-16T13:02:02.856527+0000","last_fullsized":"2024-09-16T13:02:02.856527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"142'433","reported_seq":4984,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829498+0000","last_change":"2024-09-16T13:01:38.020672+0000","last_active":"2024-09-16T13:02:02.829498+0000","last_peered":"2024-09-16T13:02:02.829498+0000","last_clean":"2024-09-16T13:02:02.829498+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:02:02.829498+0000","last_undegraded":"2024-09-16T13:02:02.829498+0000","last_fullsized":"2024-09-16T13:02:02.829498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"151'1171","reported_seq":8013,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863870+0000","last_change":"2024-09-16T13:01:38.017984+0000","last_active":"2024-09-16T13:02:02.863870+0000","last_peered":"2024-09-16T13:02:02.863870+0000","last_clean":"2024-09-16T13:02:02.863870+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:02:02.863870+0000","last_undegraded":"2024-09-16T13:02:02.863870+0000","last_fullsized":"2024-09-16T13:02:02.863870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"150'493","reported_seq":22038,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817747+0000","last_change":"2024-09-16T13:01:38.026576+0000","last_active":"2024-09-16T13:02:01.817747+0000","last_peered":"2024-09-16T13:02:01.817747+0000","last_clean":"2024-09-16T13:02:01.817747+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:02:01.817747+0000","last_undegraded":"2024-09-16T13:02:01.817747+0000","last_fullsized":"2024-09-16T13:02:01.817747+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:04.244300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"150'685","reported_seq":7141,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817710+0000","last_change":"2024-09-16T13:01:38.026337+0000","last_active":"2024-09-16T13:02:01.817710+0000","last_peered":"2024-09-16T13:02:01.817710+0000","last_clean":"2024-09-16T13:02:01.817710+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:02:01.817710+0000","last_undegraded":"2024-09-16T13:02:01.817710+0000","last_fullsized":"2024-09-16T13:02:01.817710+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:30.964357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"149'399","reported_seq":5474,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828856+0000","last_change":"2024-09-16T13:01:38.021608+0000","last_active":"2024-09-16T13:02:02.828856+0000","last_peered":"2024-09-16T13:02:02.828856+0000","last_clean":"2024-09-16T13:02:02.828856+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:02:02.828856+0000","last_undegraded":"2024-09-16T13:02:02.828856+0000","last_fullsized":"2024-09-16T13:02:02.828856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"150'440","reported_seq":6455,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817051+0000","last_change":"2024-09-16T13:01:38.024730+0000","last_active":"2024-09-16T13:02:01.817051+0000","last_peered":"2024-09-16T13:02:01.817051+0000","last_clean":"2024-09-16T13:02:01.817051+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:02:01.817051+0000","last_undegraded":"2024-09-16T13:02:01.817051+0000","last_fullsized":"2024-09-16T13:02:01.817051+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:23.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"150'889","reported_seq":7631,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829608+0000","last_change":"2024-09-16T13:01:38.021503+0000","last_active":"2024-09-16T13:02:02.829608+0000","last_peered":"2024-09-16T13:02:02.829608+0000","last_clean":"2024-09-16T13:02:02.829608+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:02:02.829608+0000","last_undegraded":"2024-09-16T13:02:02.829608+0000","last_fullsized":"2024-09-16T13:02:02.829608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"150'508","reported_seq":3310,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856528+0000","last_change":"2024-09-16T13:01:38.062149+0000","last_active":"2024-09-16T13:02:02.856528+0000","last_peered":"2024-09-16T13:02:02.856528+0000","last_clean":"2024-09-16T13:02:02.856528+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:02:02.856528+0000","last_undegraded":"2024-09-16T13:02:02.856528+0000","last_fullsized":"2024-09-16T13:02:02.856528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"150'523","reported_seq":16214,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829042+0000","last_change":"2024-09-16T13:01:38.021224+0000","last_active":"2024-09-16T13:02:02.829042+0000","last_peered":"2024-09-16T13:02:02.829042+0000","last_clean":"2024-09-16T13:02:02.829042+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:02:02.829042+0000","last_undegraded":"2024-09-16T13:02:02.829042+0000","last_fullsized":"2024-09-16T13:02:02.829042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"149'579","reported_seq":23227,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829478+0000","last_change":"2024-09-16T13:01:38.020686+0000","last_active":"2024-09-16T13:02:02.829478+0000","last_peered":"2024-09-16T13:02:02.829478+0000","last_clean":"2024-09-16T13:02:02.829478+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:02:02.829478+0000","last_undegraded":"2024-09-16T13:02:02.829478+0000","last_fullsized":"2024-09-16T13:02:02.829478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"151'512","reported_seq":6231,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817958+0000","last_change":"2024-09-16T13:01:38.027017+0000","last_active":"2024-09-16T13:02:01.817958+0000","last_peered":"2024-09-16T13:02:01.817958+0000","last_clean":"2024-09-16T13:02:01.817958+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:02:01.817958+0000","last_undegraded":"2024-09-16T13:02:01.817958+0000","last_fullsized":"2024-09-16T13:02:01.817958+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:56.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"150'371","reported_seq":3123,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828778+0000","last_change":"2024-09-16T13:01:38.019785+0000","last_active":"2024-09-16T13:02:02.828778+0000","last_peered":"2024-09-16T13:02:02.828778+0000","last_clean":"2024-09-16T13:02:02.828778+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:02:02.828778+0000","last_undegraded":"2024-09-16T13:02:02.828778+0000","last_fullsized":"2024-09-16T13:02:02.828778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"149'363","reported_seq":18827,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828605+0000","last_change":"2024-09-16T13:01:38.019836+0000","last_active":"2024-09-16T13:02:02.828605+0000","last_peered":"2024-09-16T13:02:02.828605+0000","last_clean":"2024-09-16T13:02:02.828605+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:02:02.828605+0000","last_undegraded":"2024-09-16T13:02:02.828605+0000","last_fullsized":"2024-09-16T13:02:02.828605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"139'335","reported_seq":5458,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.018514+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865667+0000","last_change":"2024-09-16T13:01:35.025514+0000","last_active":"2024-09-16T13:02:02.865667+0000","last_peered":"2024-09-16T13:02:02.865667+0000","last_clean":"2024-09-16T13:02:02.865667+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:02:02.865667+0000","last_undegraded":"2024-09-16T13:02:02.865667+0000","last_fullsized":"2024-09-16T13:02:02.865667+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:27.902219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"146'451","reported_seq":3774,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817602+0000","last_change":"2024-09-16T13:01:38.027021+0000","last_active":"2024-09-16T13:02:01.817602+0000","last_peered":"2024-09-16T13:02:01.817602+0000","last_clean":"2024-09-16T13:02:01.817602+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:02:01.817602+0000","last_undegraded":"2024-09-16T13:02:01.817602+0000","last_fullsized":"2024-09-16T13:02:01.817602+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460996+0000","last_change":"2024-09-16T13:01:34.980687+0000","last_active":"2024-09-16T13:02:02.460996+0000","last_peered":"2024-09-16T13:02:02.460996+0000","last_clean":"2024-09-16T13:02:02.460996+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:02:02.460996+0000","last_undegraded":"2024-09-16T13:02:02.460996+0000","last_fullsized":"2024-09-16T13:02:02.460996+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:44.616572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"149'690","reported_seq":7327,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.020063+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":160,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459793+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:02:02.459793+0000","last_peered":"2024-09-16T13:02:02.459793+0000","last_clean":"2024-09-16T13:02:02.459793+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:02:02.459793+0000","last_undegraded":"2024-09-16T13:02:02.459793+0000","last_fullsized":"2024-09-16T13:02:02.459793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"151'687","reported_seq":3219,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856470+0000","last_change":"2024-09-16T13:01:38.061542+0000","last_active":"2024-09-16T13:02:02.856470+0000","last_peered":"2024-09-16T13:02:02.856470+0000","last_clean":"2024-09-16T13:02:02.856470+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:02:02.856470+0000","last_undegraded":"2024-09-16T13:02:02.856470+0000","last_fullsized":"2024-09-16T13:02:02.856470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460629+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:02:02.460629+0000","last_peered":"2024-09-16T13:02:02.460629+0000","last_clean":"2024-09-16T13:02:02.460629+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:02:02.460629+0000","last_undegraded":"2024-09-16T13:02:02.460629+0000","last_fullsized":"2024-09-16T13:02:02.460629+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"150'586","reported_seq":17109,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460622+0000","last_change":"2024-09-16T13:01:38.114806+0000","last_active":"2024-09-16T13:02:02.460622+0000","last_peered":"2024-09-16T13:02:02.460622+0000","last_clean":"2024-09-16T13:02:02.460622+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:02:02.460622+0000","last_undegraded":"2024-09-16T13:02:02.460622+0000","last_fullsized":"2024-09-16T13:02:02.460622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":123,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817718+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:02:01.817718+0000","last_peered":"2024-09-16T13:02:01.817718+0000","last_clean":"2024-09-16T13:02:01.817718+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:02:01.817718+0000","last_undegraded":"2024-09-16T13:02:01.817718+0000","last_fullsized":"2024-09-16T13:02:01.817718+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":191,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829038+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:02:02.829038+0000","last_peered":"2024-09-16T13:02:02.829038+0000","last_clean":"2024-09-16T13:02:02.829038+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:02:02.829038+0000","last_undegraded":"2024-09-16T13:02:02.829038+0000","last_fullsized":"2024-09-16T13:02:02.829038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"148'501","reported_seq":17937,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865403+0000","last_change":"2024-09-16T13:01:38.031901+0000","last_active":"2024-09-16T13:02:02.865403+0000","last_peered":"2024-09-16T13:02:02.865403+0000","last_clean":"2024-09-16T13:02:02.865403+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:02:02.865403+0000","last_undegraded":"2024-09-16T13:02:02.865403+0000","last_fullsized":"2024-09-16T13:02:02.865403+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:27.630481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411569999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":121,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.460348+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:02:02.460348+0000","last_peered":"2024-09-16T13:02:02.460348+0000","last_clean":"2024-09-16T13:02:02.460348+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:02:02.460348+0000","last_undegraded":"2024-09-16T13:02:02.460348+0000","last_fullsized":"2024-09-16T13:02:02.460348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863815+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:02:02.863815+0000","last_peered":"2024-09-16T13:02:02.863815+0000","last_clean":"2024-09-16T13:02:02.863815+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:02:02.863815+0000","last_undegraded":"2024-09-16T13:02:02.863815+0000","last_fullsized":"2024-09-16T13:02:02.863815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'407","reported_seq":2774,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.461059+0000","last_change":"2024-09-16T13:01:38.113780+0000","last_active":"2024-09-16T13:02:02.461059+0000","last_peered":"2024-09-16T13:02:02.461059+0000","last_clean":"2024-09-16T13:02:02.461059+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:02:02.461059+0000","last_undegraded":"2024-09-16T13:02:02.461059+0000","last_fullsized":"2024-09-16T13:02:02.461059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":169,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856355+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:02:02.856355+0000","last_peered":"2024-09-16T13:02:02.856355+0000","last_clean":"2024-09-16T13:02:02.856355+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:02:02.856355+0000","last_undegraded":"2024-09-16T13:02:02.856355+0000","last_fullsized":"2024-09-16T13:02:02.856355+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459690+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:02:02.459690+0000","last_peered":"2024-09-16T13:02:02.459690+0000","last_clean":"2024-09-16T13:02:02.459690+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:02:02.459690+0000","last_undegraded":"2024-09-16T13:02:02.459690+0000","last_fullsized":"2024-09-16T13:02:02.459690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"151'434","reported_seq":7322,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856609+0000","last_change":"2024-09-16T13:01:38.062717+0000","last_active":"2024-09-16T13:02:02.856609+0000","last_peered":"2024-09-16T13:02:02.856609+0000","last_clean":"2024-09-16T13:02:02.856609+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:02:02.856609+0000","last_undegraded":"2024-09-16T13:02:02.856609+0000","last_fullsized":"2024-09-16T13:02:02.856609+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"151'4","reported_seq":158,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829055+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:02:02.829055+0000","last_peered":"2024-09-16T13:02:02.829055+0000","last_clean":"2024-09-16T13:02:02.829055+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:02:02.829055+0000","last_undegraded":"2024-09-16T13:02:02.829055+0000","last_fullsized":"2024-09-16T13:02:02.829055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":164,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829054+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:02:02.829054+0000","last_peered":"2024-09-16T13:02:02.829054+0000","last_clean":"2024-09-16T13:02:02.829054+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:02:02.829054+0000","last_undegraded":"2024-09-16T13:02:02.829054+0000","last_fullsized":"2024-09-16T13:02:02.829054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":248,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.459761+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:02:02.459761+0000","last_peered":"2024-09-16T13:02:02.459761+0000","last_clean":"2024-09-16T13:02:02.459761+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:02:02.459761+0000","last_undegraded":"2024-09-16T13:02:02.459761+0000","last_fullsized":"2024-09-16T13:02:02.459761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"150'444","reported_seq":3720,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865541+0000","last_change":"2024-09-16T13:01:38.031044+0000","last_active":"2024-09-16T13:02:02.865541+0000","last_peered":"2024-09-16T13:02:02.865541+0000","last_clean":"2024-09-16T13:02:02.865541+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:02:02.865541+0000","last_undegraded":"2024-09-16T13:02:02.865541+0000","last_fullsized":"2024-09-16T13:02:02.865541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:16.451880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602565,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"150'977","reported_seq":18782,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828748+0000","last_change":"2024-09-16T13:01:38.019797+0000","last_active":"2024-09-16T13:02:02.828748+0000","last_peered":"2024-09-16T13:02:02.828748+0000","last_clean":"2024-09-16T13:02:02.828748+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:02:02.828748+0000","last_undegraded":"2024-09-16T13:02:02.828748+0000","last_fullsized":"2024-09-16T13:02:02.828748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"149'328","reported_seq":846,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817233+0000","last_change":"2024-09-16T13:01:38.024649+0000","last_active":"2024-09-16T13:02:01.817233+0000","last_peered":"2024-09-16T13:02:01.817233+0000","last_clean":"2024-09-16T13:02:01.817233+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:02:01.817233+0000","last_undegraded":"2024-09-16T13:02:01.817233+0000","last_fullsized":"2024-09-16T13:02:01.817233+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"150'1380","reported_seq":6972,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865373+0000","last_change":"2024-09-16T13:01:38.031686+0000","last_active":"2024-09-16T13:02:02.865373+0000","last_peered":"2024-09-16T13:02:02.865373+0000","last_clean":"2024-09-16T13:02:02.865373+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:02:02.865373+0000","last_undegraded":"2024-09-16T13:02:02.865373+0000","last_fullsized":"2024-09-16T13:02:02.865373+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:00.052447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093887699999999996,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"144'974","reported_seq":6637,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856424+0000","last_change":"2024-09-16T13:01:38.060958+0000","last_active":"2024-09-16T13:02:02.856424+0000","last_peered":"2024-09-16T13:02:02.856424+0000","last_clean":"2024-09-16T13:02:02.856424+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:02:02.856424+0000","last_undegraded":"2024-09-16T13:02:02.856424+0000","last_fullsized":"2024-09-16T13:02:02.856424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"151'553","reported_seq":5349,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817849+0000","last_change":"2024-09-16T13:01:38.025883+0000","last_active":"2024-09-16T13:02:01.817849+0000","last_peered":"2024-09-16T13:02:01.817849+0000","last_clean":"2024-09-16T13:02:01.817849+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:02:01.817849+0000","last_undegraded":"2024-09-16T13:02:01.817849+0000","last_fullsized":"2024-09-16T13:02:01.817849+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"150'434","reported_seq":4404,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.817961+0000","last_change":"2024-09-16T13:01:38.027791+0000","last_active":"2024-09-16T13:02:01.817961+0000","last_peered":"2024-09-16T13:02:01.817961+0000","last_clean":"2024-09-16T13:02:01.817961+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:02:01.817961+0000","last_undegraded":"2024-09-16T13:02:01.817961+0000","last_fullsized":"2024-09-16T13:02:01.817961+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"150'876","reported_seq":4050,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:01.818018+0000","last_change":"2024-09-16T13:01:38.025990+0000","last_active":"2024-09-16T13:02:01.818018+0000","last_peered":"2024-09-16T13:02:01.818018+0000","last_clean":"2024-09-16T13:02:01.818018+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:02:01.818018+0000","last_undegraded":"2024-09-16T13:02:01.818018+0000","last_fullsized":"2024-09-16T13:02:01.818018+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:12.558678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"149'919","reported_seq":36748,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.461057+0000","last_change":"2024-09-16T13:01:38.113695+0000","last_active":"2024-09-16T13:02:02.461057+0000","last_peered":"2024-09-16T13:02:02.461057+0000","last_clean":"2024-09-16T13:02:02.461057+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:02:02.461057+0000","last_undegraded":"2024-09-16T13:02:02.461057+0000","last_fullsized":"2024-09-16T13:02:02.461057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21015,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":19195,"ondisk_log_size":19195,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192521,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301756,"kb_used_data":2028,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070084,"statfs":{"total":96636764160,"available":96327766016,"internally_reserved":0,"allocated":2076672,"data_stored":1051715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]}]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303568,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068272,"statfs":{"total":96636764160,"available":96325910528,"internally_reserved":0,"allocated":2088960,"data_stored":1051902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.85599999999999998},"min":{"1min":0.48099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1499999999999999,"5min":1.272,"15min":1.272},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.83999999999999997},"min":{"1min":0.55100000000000005,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.391,"15min":1.391},"last":0.57899999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.97299999999999998,"15min":0.89200000000000002},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.333,"5min":14.275,"15min":14.275},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.58699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3600000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.752}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.58099999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2430000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.996,"5min":1.071,"15min":1.103},"min":{"1min":0.61799999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4830000000000001,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.95299999999999996}]},{"osd":3,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.071,"15min":1.0669999999999999},"min":{"1min":0.61899999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.2989999999999999,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.115},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6299999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.272}]},{"osd":4,"last update":"Mon Sep 16 13:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0109999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":1.446},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.077,"15min":1.137},"min":{"1min":0.65300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4379999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.29},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1850000000000001,"15min":1.2450000000000001},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.673,"5min":1.748,"15min":1.748},"last":1.369}]}]},{"osd":4,"up_from":108,"seq":463856467984,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309436,"kb_used_data":2020,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062404,"statfs":{"total":96636764160,"available":96319901696,"internally_reserved":0,"allocated":2068480,"data_stored":1045486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.0309999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.3899999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.23},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.4630000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.4339999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]},{"osd":7,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.498}]}]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299392,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072448,"statfs":{"total":96636764160,"available":96330186752,"internally_reserved":0,"allocated":2076672,"data_stored":1045654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.995,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71799999999999997,"15min":0.66700000000000004},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0940000000000001,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.232,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.873},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.51500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.103,"5min":1.528,"15min":1.528},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98399999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.49299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5740000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.0529999999999999}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.165}]},{"osd":5,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.109,"15min":1.129},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8520000000000001,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.171,"5min":1.1060000000000001,"15min":1.1180000000000001},"min":{"1min":0.56000000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.143}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1299999999999999},"min":{"1min":0.61099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.363},{"interface":"front","average":{"1min":1.167,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.77700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9830000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.304}]}]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299388,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072452,"statfs":{"total":96636764160,"available":96330190848,"internally_reserved":0,"allocated":2072576,"data_stored":1051222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.314,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.0980000000000001,"5min":10.654,"15min":10.654},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.77100000000000002},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.97899999999999998,"5min":10.816000000000001,"15min":10.816000000000001},"last":1.2589999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.52200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99099999999999999,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.97999999999999998}]},{"osd":3,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.89200000000000002,"15min":0.80400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.2430000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.077,"5min":1.647,"15min":1.647},"last":0.873}]},{"osd":4,"last update":"Mon Sep 16 13:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.365}]},{"osd":5,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.093,"15min":1.1930000000000001},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.226,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.516},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0740000000000001,"15min":1.1439999999999999},"min":{"1min":0.60799999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.383,"5min":2.125,"15min":2.125},"last":1.772}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.8089999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.681}]},{"osd":7,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.139,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.478,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.7350000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.1539999999999999,"15min":1.2450000000000001},"min":{"1min":0.63700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.538,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.901}]}]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303600,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068240,"statfs":{"total":96636764160,"available":96325877760,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.88600000000000001,"15min":0.84199999999999997},"min":{"1min":0.39100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.1899999999999999,"5min":9.407,"15min":9.407},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.85999999999999999,"15min":0.94599999999999995},"min":{"1min":0.46899999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.319,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0469999999999999,"5min":2.254,"15min":2.254},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.36399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83199999999999996,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":1.028},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4750000000000001,"5min":2.044,"15min":2.044},"last":0.879}]},{"osd":4,"last update":"Mon Sep 16 13:01:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.038},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.177}]},{"osd":5,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0149999999999999,"15min":1.0640000000000001},"min":{"1min":0.56399999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4730000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.999,"5min":1.133,"15min":1.2050000000000001},"min":{"1min":0.60999999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.325,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.125}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.145}]},{"osd":7,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.129,"15min":1.177},"min":{"1min":0.745,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.327,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.093},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1599999999999999},"min":{"1min":0.70099999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.427,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.2010000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303596,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068244,"statfs":{"total":96636764160,"available":96325881856,"internally_reserved":0,"allocated":2121728,"data_stored":1635835,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5920000000000001,"5min":1.73,"15min":1.73},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.145,"15min":1.26},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3080000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.98199999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0249999999999999,"15min":1.1559999999999999},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.208,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.56499999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5149999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.81999999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0569999999999999},"min":{"1min":0.55400000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.298,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.143,"15min":1.2030000000000001},"min":{"1min":0.50900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1839999999999999,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.0189999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0109999999999999,"15min":1.0720000000000001},"min":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.984,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.117},{"interface":"front","average":{"1min":1.028,"5min":1.2010000000000001,"15min":1.298},"min":{"1min":0.66900000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9299999999999999,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.272}]},{"osd":4,"last update":"Mon Sep 16 13:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.046},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.96199999999999997}]},{"osd":5,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.1759999999999999,"15min":1.3029999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.135,"5min":6.282,"15min":6.282},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.93600000000000005,"15min":0.97799999999999998},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.2549999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.153}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.083},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.349}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303536,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068304,"statfs":{"total":96636764160,"available":96325943296,"internally_reserved":0,"allocated":2068480,"data_stored":1045486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.73999999999999999},"min":{"1min":0.38100000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.97799999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.254,"15min":0.254},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72799999999999998,"15min":0.77600000000000002},"min":{"1min":0.35699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.90400000000000003,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.181,"5min":3.024,"15min":3.024},"last":1.1990000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.95999999999999996,"15min":1.034},"min":{"1min":0.53600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2250000000000001,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.49199999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.151,"5min":3.544,"15min":3.544},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.034},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.25,"15min":1.3979999999999999},"min":{"1min":0.55700000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.619,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0600000000000001,"15min":1.0840000000000001},"min":{"1min":0.78700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.423,"5min":2.5,"15min":2.5},"last":1.238}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.367},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.49}]},{"osd":7,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1279999999999999,"15min":1.1459999999999999},"min":{"1min":0.879,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.575,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.139,"15min":1.181},"min":{"1min":0.68700000000000006,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.534,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.5980000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:02:10.153 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T13:02:10.154 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:02:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:02:10.471 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-16T13:02:10.291084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":1005,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21120,"num_write_kb":309526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19275,"ondisk_log_size":19275,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424292,"kb_used_data":16352,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752550428,"statfs":{"total":773094113280,"available":770611638272,"internally_reserved":0,"allocated":16744448,"data_stored":9588127,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002414"},"pg_stats":[{"pgid":"4.1f","version":"151'339","reported_seq":4229,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865786+0000","last_change":"2024-09-16T13:01:38.032360+0000","last_active":"2024-09-16T13:02:02.865786+0000","last_peered":"2024-09-16T13:02:02.865786+0000","last_clean":"2024-09-16T13:02:02.865786+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:02:02.865786+0000","last_undegraded":"2024-09-16T13:02:02.865786+0000","last_fullsized":"2024-09-16T13:02:02.865786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:11.756487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015622850000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":333,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"150'514","reported_seq":17234,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856527+0000","last_change":"2024-09-16T13:01:38.062519+0000","last_active":"2024-09-16T13:02:02.856527+0000","last_peered":"2024-09-16T13:02:02.856527+0000","last_clean":"2024-09-16T13:02:02.856527+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:02:02.856527+0000","last_undegraded":"2024-09-16T13:02:02.856527+0000","last_fullsized":"2024-09-16T13:02:02.856527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"142'433","reported_seq":4984,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829498+0000","last_change":"2024-09-16T13:01:38.020672+0000","last_active":"2024-09-16T13:02:02.829498+0000","last_peered":"2024-09-16T13:02:02.829498+0000","last_clean":"2024-09-16T13:02:02.829498+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:02:02.829498+0000","last_undegraded":"2024-09-16T13:02:02.829498+0000","last_fullsized":"2024-09-16T13:02:02.829498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"151'1171","reported_seq":8013,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863870+0000","last_change":"2024-09-16T13:01:38.017984+0000","last_active":"2024-09-16T13:02:02.863870+0000","last_peered":"2024-09-16T13:02:02.863870+0000","last_clean":"2024-09-16T13:02:02.863870+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:02:02.863870+0000","last_undegraded":"2024-09-16T13:02:02.863870+0000","last_fullsized":"2024-09-16T13:02:02.863870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"150'493","reported_seq":22039,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827723+0000","last_change":"2024-09-16T13:01:38.026576+0000","last_active":"2024-09-16T13:02:02.827723+0000","last_peered":"2024-09-16T13:02:02.827723+0000","last_clean":"2024-09-16T13:02:02.827723+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:02:02.827723+0000","last_undegraded":"2024-09-16T13:02:02.827723+0000","last_fullsized":"2024-09-16T13:02:02.827723+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:04.244300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"150'685","reported_seq":7142,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828217+0000","last_change":"2024-09-16T13:01:38.026337+0000","last_active":"2024-09-16T13:02:02.828217+0000","last_peered":"2024-09-16T13:02:02.828217+0000","last_clean":"2024-09-16T13:02:02.828217+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:02:02.828217+0000","last_undegraded":"2024-09-16T13:02:02.828217+0000","last_fullsized":"2024-09-16T13:02:02.828217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:30.964357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"149'399","reported_seq":5474,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828856+0000","last_change":"2024-09-16T13:01:38.021608+0000","last_active":"2024-09-16T13:02:02.828856+0000","last_peered":"2024-09-16T13:02:02.828856+0000","last_clean":"2024-09-16T13:02:02.828856+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:02:02.828856+0000","last_undegraded":"2024-09-16T13:02:02.828856+0000","last_fullsized":"2024-09-16T13:02:02.828856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"150'440","reported_seq":6456,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827434+0000","last_change":"2024-09-16T13:01:38.024730+0000","last_active":"2024-09-16T13:02:02.827434+0000","last_peered":"2024-09-16T13:02:02.827434+0000","last_clean":"2024-09-16T13:02:02.827434+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:02:02.827434+0000","last_undegraded":"2024-09-16T13:02:02.827434+0000","last_fullsized":"2024-09-16T13:02:02.827434+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:23.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"150'889","reported_seq":7631,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829608+0000","last_change":"2024-09-16T13:01:38.021503+0000","last_active":"2024-09-16T13:02:02.829608+0000","last_peered":"2024-09-16T13:02:02.829608+0000","last_clean":"2024-09-16T13:02:02.829608+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:02:02.829608+0000","last_undegraded":"2024-09-16T13:02:02.829608+0000","last_fullsized":"2024-09-16T13:02:02.829608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"150'508","reported_seq":3310,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856528+0000","last_change":"2024-09-16T13:01:38.062149+0000","last_active":"2024-09-16T13:02:02.856528+0000","last_peered":"2024-09-16T13:02:02.856528+0000","last_clean":"2024-09-16T13:02:02.856528+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:02:02.856528+0000","last_undegraded":"2024-09-16T13:02:02.856528+0000","last_fullsized":"2024-09-16T13:02:02.856528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"150'523","reported_seq":16214,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829042+0000","last_change":"2024-09-16T13:01:38.021224+0000","last_active":"2024-09-16T13:02:02.829042+0000","last_peered":"2024-09-16T13:02:02.829042+0000","last_clean":"2024-09-16T13:02:02.829042+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:02:02.829042+0000","last_undegraded":"2024-09-16T13:02:02.829042+0000","last_fullsized":"2024-09-16T13:02:02.829042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"149'579","reported_seq":23227,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829478+0000","last_change":"2024-09-16T13:01:38.020686+0000","last_active":"2024-09-16T13:02:02.829478+0000","last_peered":"2024-09-16T13:02:02.829478+0000","last_clean":"2024-09-16T13:02:02.829478+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:02:02.829478+0000","last_undegraded":"2024-09-16T13:02:02.829478+0000","last_fullsized":"2024-09-16T13:02:02.829478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"151'512","reported_seq":6232,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827930+0000","last_change":"2024-09-16T13:01:38.027017+0000","last_active":"2024-09-16T13:02:02.827930+0000","last_peered":"2024-09-16T13:02:02.827930+0000","last_clean":"2024-09-16T13:02:02.827930+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:02:02.827930+0000","last_undegraded":"2024-09-16T13:02:02.827930+0000","last_fullsized":"2024-09-16T13:02:02.827930+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:56.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"150'371","reported_seq":3123,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828778+0000","last_change":"2024-09-16T13:01:38.019785+0000","last_active":"2024-09-16T13:02:02.828778+0000","last_peered":"2024-09-16T13:02:02.828778+0000","last_clean":"2024-09-16T13:02:02.828778+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:02:02.828778+0000","last_undegraded":"2024-09-16T13:02:02.828778+0000","last_fullsized":"2024-09-16T13:02:02.828778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"149'363","reported_seq":18827,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828605+0000","last_change":"2024-09-16T13:01:38.019836+0000","last_active":"2024-09-16T13:02:02.828605+0000","last_peered":"2024-09-16T13:02:02.828605+0000","last_clean":"2024-09-16T13:02:02.828605+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:02:02.828605+0000","last_undegraded":"2024-09-16T13:02:02.828605+0000","last_fullsized":"2024-09-16T13:02:02.828605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"139'335","reported_seq":5458,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.018514+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865667+0000","last_change":"2024-09-16T13:01:35.025514+0000","last_active":"2024-09-16T13:02:02.865667+0000","last_peered":"2024-09-16T13:02:02.865667+0000","last_clean":"2024-09-16T13:02:02.865667+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:02:02.865667+0000","last_undegraded":"2024-09-16T13:02:02.865667+0000","last_fullsized":"2024-09-16T13:02:02.865667+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:27.902219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"146'451","reported_seq":3775,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828186+0000","last_change":"2024-09-16T13:01:38.027021+0000","last_active":"2024-09-16T13:02:02.828186+0000","last_peered":"2024-09-16T13:02:02.828186+0000","last_clean":"2024-09-16T13:02:02.828186+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:02:02.828186+0000","last_undegraded":"2024-09-16T13:02:02.828186+0000","last_fullsized":"2024-09-16T13:02:02.828186+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.864188+0000","last_change":"2024-09-16T13:01:34.980687+0000","last_active":"2024-09-16T13:02:02.864188+0000","last_peered":"2024-09-16T13:02:02.864188+0000","last_clean":"2024-09-16T13:02:02.864188+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:02:02.864188+0000","last_undegraded":"2024-09-16T13:02:02.864188+0000","last_fullsized":"2024-09-16T13:02:02.864188+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:44.616572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"149'690","reported_seq":7327,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.020063+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":161,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865756+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:02:02.865756+0000","last_peered":"2024-09-16T13:02:02.865756+0000","last_clean":"2024-09-16T13:02:02.865756+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:02:02.865756+0000","last_undegraded":"2024-09-16T13:02:02.865756+0000","last_fullsized":"2024-09-16T13:02:02.865756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"151'687","reported_seq":3219,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856470+0000","last_change":"2024-09-16T13:01:38.061542+0000","last_active":"2024-09-16T13:02:02.856470+0000","last_peered":"2024-09-16T13:02:02.856470+0000","last_clean":"2024-09-16T13:02:02.856470+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:02:02.856470+0000","last_undegraded":"2024-09-16T13:02:02.856470+0000","last_fullsized":"2024-09-16T13:02:02.856470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":122,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.864151+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:02:02.864151+0000","last_peered":"2024-09-16T13:02:02.864151+0000","last_clean":"2024-09-16T13:02:02.864151+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:02:02.864151+0000","last_undegraded":"2024-09-16T13:02:02.864151+0000","last_fullsized":"2024-09-16T13:02:02.864151+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"150'586","reported_seq":17110,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.864374+0000","last_change":"2024-09-16T13:01:38.114806+0000","last_active":"2024-09-16T13:02:02.864374+0000","last_peered":"2024-09-16T13:02:02.864374+0000","last_clean":"2024-09-16T13:02:02.864374+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:02:02.864374+0000","last_undegraded":"2024-09-16T13:02:02.864374+0000","last_fullsized":"2024-09-16T13:02:02.864374+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":124,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827707+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:02:02.827707+0000","last_peered":"2024-09-16T13:02:02.827707+0000","last_clean":"2024-09-16T13:02:02.827707+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:02:02.827707+0000","last_undegraded":"2024-09-16T13:02:02.827707+0000","last_fullsized":"2024-09-16T13:02:02.827707+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":191,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829038+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:02:02.829038+0000","last_peered":"2024-09-16T13:02:02.829038+0000","last_clean":"2024-09-16T13:02:02.829038+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:02:02.829038+0000","last_undegraded":"2024-09-16T13:02:02.829038+0000","last_fullsized":"2024-09-16T13:02:02.829038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"148'501","reported_seq":17937,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865403+0000","last_change":"2024-09-16T13:01:38.031901+0000","last_active":"2024-09-16T13:02:02.865403+0000","last_peered":"2024-09-16T13:02:02.865403+0000","last_clean":"2024-09-16T13:02:02.865403+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:02:02.865403+0000","last_undegraded":"2024-09-16T13:02:02.865403+0000","last_fullsized":"2024-09-16T13:02:02.865403+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:27.630481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411569999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":122,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863991+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:02:02.863991+0000","last_peered":"2024-09-16T13:02:02.863991+0000","last_clean":"2024-09-16T13:02:02.863991+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:02:02.863991+0000","last_undegraded":"2024-09-16T13:02:02.863991+0000","last_fullsized":"2024-09-16T13:02:02.863991+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863815+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:02:02.863815+0000","last_peered":"2024-09-16T13:02:02.863815+0000","last_clean":"2024-09-16T13:02:02.863815+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:02:02.863815+0000","last_undegraded":"2024-09-16T13:02:02.863815+0000","last_fullsized":"2024-09-16T13:02:02.863815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'407","reported_seq":2775,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.864304+0000","last_change":"2024-09-16T13:01:38.113780+0000","last_active":"2024-09-16T13:02:02.864304+0000","last_peered":"2024-09-16T13:02:02.864304+0000","last_clean":"2024-09-16T13:02:02.864304+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:02:02.864304+0000","last_undegraded":"2024-09-16T13:02:02.864304+0000","last_fullsized":"2024-09-16T13:02:02.864304+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":169,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856355+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:02:02.856355+0000","last_peered":"2024-09-16T13:02:02.856355+0000","last_clean":"2024-09-16T13:02:02.856355+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:02:02.856355+0000","last_undegraded":"2024-09-16T13:02:02.856355+0000","last_fullsized":"2024-09-16T13:02:02.856355+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":179,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865704+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:02:02.865704+0000","last_peered":"2024-09-16T13:02:02.865704+0000","last_clean":"2024-09-16T13:02:02.865704+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:02:02.865704+0000","last_undegraded":"2024-09-16T13:02:02.865704+0000","last_fullsized":"2024-09-16T13:02:02.865704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"151'434","reported_seq":7322,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856609+0000","last_change":"2024-09-16T13:01:38.062717+0000","last_active":"2024-09-16T13:02:02.856609+0000","last_peered":"2024-09-16T13:02:02.856609+0000","last_clean":"2024-09-16T13:02:02.856609+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:02:02.856609+0000","last_undegraded":"2024-09-16T13:02:02.856609+0000","last_fullsized":"2024-09-16T13:02:02.856609+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"151'4","reported_seq":158,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829055+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:02:02.829055+0000","last_peered":"2024-09-16T13:02:02.829055+0000","last_clean":"2024-09-16T13:02:02.829055+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:02:02.829055+0000","last_undegraded":"2024-09-16T13:02:02.829055+0000","last_fullsized":"2024-09-16T13:02:02.829055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":164,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829054+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:02:02.829054+0000","last_peered":"2024-09-16T13:02:02.829054+0000","last_clean":"2024-09-16T13:02:02.829054+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:02:02.829054+0000","last_undegraded":"2024-09-16T13:02:02.829054+0000","last_fullsized":"2024-09-16T13:02:02.829054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":249,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865907+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:02:02.865907+0000","last_peered":"2024-09-16T13:02:02.865907+0000","last_clean":"2024-09-16T13:02:02.865907+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:02:02.865907+0000","last_undegraded":"2024-09-16T13:02:02.865907+0000","last_fullsized":"2024-09-16T13:02:02.865907+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"150'444","reported_seq":3720,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865541+0000","last_change":"2024-09-16T13:01:38.031044+0000","last_active":"2024-09-16T13:02:02.865541+0000","last_peered":"2024-09-16T13:02:02.865541+0000","last_clean":"2024-09-16T13:02:02.865541+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:02:02.865541+0000","last_undegraded":"2024-09-16T13:02:02.865541+0000","last_fullsized":"2024-09-16T13:02:02.865541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:16.451880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602565,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"150'977","reported_seq":18782,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828748+0000","last_change":"2024-09-16T13:01:38.019797+0000","last_active":"2024-09-16T13:02:02.828748+0000","last_peered":"2024-09-16T13:02:02.828748+0000","last_clean":"2024-09-16T13:02:02.828748+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:02:02.828748+0000","last_undegraded":"2024-09-16T13:02:02.828748+0000","last_fullsized":"2024-09-16T13:02:02.828748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"149'328","reported_seq":847,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827607+0000","last_change":"2024-09-16T13:01:38.024649+0000","last_active":"2024-09-16T13:02:02.827607+0000","last_peered":"2024-09-16T13:02:02.827607+0000","last_clean":"2024-09-16T13:02:02.827607+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:02:02.827607+0000","last_undegraded":"2024-09-16T13:02:02.827607+0000","last_fullsized":"2024-09-16T13:02:02.827607+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"150'1380","reported_seq":6972,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865373+0000","last_change":"2024-09-16T13:01:38.031686+0000","last_active":"2024-09-16T13:02:02.865373+0000","last_peered":"2024-09-16T13:02:02.865373+0000","last_clean":"2024-09-16T13:02:02.865373+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:02:02.865373+0000","last_undegraded":"2024-09-16T13:02:02.865373+0000","last_fullsized":"2024-09-16T13:02:02.865373+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:00.052447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093887699999999996,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"144'974","reported_seq":6637,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856424+0000","last_change":"2024-09-16T13:01:38.060958+0000","last_active":"2024-09-16T13:02:02.856424+0000","last_peered":"2024-09-16T13:02:02.856424+0000","last_clean":"2024-09-16T13:02:02.856424+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:02:02.856424+0000","last_undegraded":"2024-09-16T13:02:02.856424+0000","last_fullsized":"2024-09-16T13:02:02.856424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"151'553","reported_seq":5350,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827827+0000","last_change":"2024-09-16T13:01:38.025883+0000","last_active":"2024-09-16T13:02:02.827827+0000","last_peered":"2024-09-16T13:02:02.827827+0000","last_clean":"2024-09-16T13:02:02.827827+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:02:02.827827+0000","last_undegraded":"2024-09-16T13:02:02.827827+0000","last_fullsized":"2024-09-16T13:02:02.827827+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"150'434","reported_seq":4405,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827901+0000","last_change":"2024-09-16T13:01:38.027791+0000","last_active":"2024-09-16T13:02:02.827901+0000","last_peered":"2024-09-16T13:02:02.827901+0000","last_clean":"2024-09-16T13:02:02.827901+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:02:02.827901+0000","last_undegraded":"2024-09-16T13:02:02.827901+0000","last_fullsized":"2024-09-16T13:02:02.827901+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"150'876","reported_seq":4051,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827980+0000","last_change":"2024-09-16T13:01:38.025990+0000","last_active":"2024-09-16T13:02:02.827980+0000","last_peered":"2024-09-16T13:02:02.827980+0000","last_clean":"2024-09-16T13:02:02.827980+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:02:02.827980+0000","last_undegraded":"2024-09-16T13:02:02.827980+0000","last_fullsized":"2024-09-16T13:02:02.827980+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:12.558678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"149'919","reported_seq":36749,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.864270+0000","last_change":"2024-09-16T13:01:38.113695+0000","last_active":"2024-09-16T13:02:02.864270+0000","last_peered":"2024-09-16T13:02:02.864270+0000","last_clean":"2024-09-16T13:02:02.864270+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:02:02.864270+0000","last_undegraded":"2024-09-16T13:02:02.864270+0000","last_fullsized":"2024-09-16T13:02:02.864270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21015,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":19195,"ondisk_log_size":19195,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192521,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301756,"kb_used_data":2028,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070084,"statfs":{"total":96636764160,"available":96327766016,"internally_reserved":0,"allocated":2076672,"data_stored":1051715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]}]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303568,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068272,"statfs":{"total":96636764160,"available":96325910528,"internally_reserved":0,"allocated":2088960,"data_stored":1051902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.85599999999999998},"min":{"1min":0.48099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1499999999999999,"5min":1.272,"15min":1.272},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.83999999999999997},"min":{"1min":0.55100000000000005,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.391,"15min":1.391},"last":0.57899999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.97299999999999998,"15min":0.89200000000000002},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.333,"5min":14.275,"15min":14.275},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.58699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3600000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.752}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.58099999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2430000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.996,"5min":1.071,"15min":1.103},"min":{"1min":0.61799999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4830000000000001,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.95299999999999996}]},{"osd":3,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.071,"15min":1.0669999999999999},"min":{"1min":0.61899999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.2989999999999999,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.115},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6299999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.272}]},{"osd":4,"last update":"Mon Sep 16 13:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0109999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":1.446},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.077,"15min":1.137},"min":{"1min":0.65300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4379999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.29},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1850000000000001,"15min":1.2450000000000001},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.673,"5min":1.748,"15min":1.748},"last":1.369}]}]},{"osd":4,"up_from":108,"seq":463856467985,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309436,"kb_used_data":2032,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062404,"statfs":{"total":96636764160,"available":96319901696,"internally_reserved":0,"allocated":2080768,"data_stored":1051734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.83799999999999997}]},{"osd":1,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.92100000000000004}]},{"osd":2,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.80700000000000005}]},{"osd":5,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":7,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.876},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.97299999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299392,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072448,"statfs":{"total":96636764160,"available":96330186752,"internally_reserved":0,"allocated":2076672,"data_stored":1045654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.995,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71799999999999997,"15min":0.66700000000000004},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0940000000000001,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.232,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.873},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.51500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.103,"5min":1.528,"15min":1.528},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98399999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.49299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5740000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.0529999999999999}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.165}]},{"osd":5,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.109,"15min":1.129},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8520000000000001,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.171,"5min":1.1060000000000001,"15min":1.1180000000000001},"min":{"1min":0.56000000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.143}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1299999999999999},"min":{"1min":0.61099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.363},{"interface":"front","average":{"1min":1.167,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.77700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9830000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.304}]}]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299388,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072452,"statfs":{"total":96636764160,"available":96330190848,"internally_reserved":0,"allocated":2072576,"data_stored":1051222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.314,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.0980000000000001,"5min":10.654,"15min":10.654},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.77100000000000002},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.97899999999999998,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.66900000000000004}]},{"osd":1,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.52200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99099999999999999,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.499}]},{"osd":3,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.89200000000000002,"15min":0.80400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.2430000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.077,"5min":1.647,"15min":1.647},"last":0.59599999999999997}]},{"osd":4,"last update":"Mon Sep 16 13:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.85999999999999999}]},{"osd":5,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.093,"15min":1.1930000000000001},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.226,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.874},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0740000000000001,"15min":1.1439999999999999},"min":{"1min":0.60799999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.383,"5min":2.125,"15min":2.125},"last":0.89200000000000002}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.749},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.139,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.478,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.054,"5min":1.1539999999999999,"15min":1.2450000000000001},"min":{"1min":0.63700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.538,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.65400000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303600,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068240,"statfs":{"total":96636764160,"available":96325877760,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.88600000000000001,"15min":0.84199999999999997},"min":{"1min":0.39100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.1899999999999999,"5min":9.407,"15min":9.407},"last":0.627},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72499999999999998}]},{"osd":2,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.85999999999999999,"15min":0.94599999999999995},"min":{"1min":0.46899999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.319,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.754},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0469999999999999,"5min":2.254,"15min":2.254},"last":0.54300000000000004}]},{"osd":3,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.36399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83199999999999996,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":1.028},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4750000000000001,"5min":2.044,"15min":2.044},"last":0.65600000000000003}]},{"osd":4,"last update":"Mon Sep 16 13:01:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.80200000000000005}]},{"osd":5,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0149999999999999,"15min":1.0640000000000001},"min":{"1min":0.56399999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4730000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.999,"5min":1.133,"15min":1.2050000000000001},"min":{"1min":0.60999999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.325,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.0169999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.1899999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.129,"15min":1.177},"min":{"1min":0.745,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.327,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1599999999999999},"min":{"1min":0.70099999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.427,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.90100000000000002}]}]},{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303600,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068240,"statfs":{"total":96636764160,"available":96325877760,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5920000000000001,"5min":1.73,"15min":1.73},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.145,"15min":1.26},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3080000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.66300000000000003}]},{"osd":1,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0249999999999999,"15min":1.1559999999999999},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.208,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.56499999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5149999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0569999999999999},"min":{"1min":0.55400000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.298,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.143,"15min":1.2030000000000001},"min":{"1min":0.50900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1839999999999999,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":0.878}]},{"osd":3,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0109999999999999,"15min":1.0720000000000001},"min":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.984,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.028,"5min":1.2010000000000001,"15min":1.298},"min":{"1min":0.66900000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9299999999999999,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":0.91900000000000004}]},{"osd":4,"last update":"Mon Sep 16 13:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.0700000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.1759999999999999,"15min":1.3029999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.135,"5min":6.282,"15min":6.282},"last":1.014},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.93600000000000005,"15min":0.97799999999999998},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.2549999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.036}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.1559999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303552,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068288,"statfs":{"total":96636764160,"available":96325926912,"internally_reserved":0,"allocated":2080768,"data_stored":1051734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.73999999999999999},"min":{"1min":0.38100000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.97799999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.254,"15min":0.254},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.75900000000000001}]},{"osd":2,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72799999999999998,"15min":0.77600000000000002},"min":{"1min":0.35699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.90400000000000003,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.181,"5min":3.024,"15min":3.024},"last":0.66600000000000004}]},{"osd":3,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.95999999999999996,"15min":1.034},"min":{"1min":0.53600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2250000000000001,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.49199999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.151,"5min":3.544,"15min":3.544},"last":0.91900000000000004}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.1879999999999999}]},{"osd":5,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.25,"15min":1.3979999999999999},"min":{"1min":0.55700000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.619,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.997},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0600000000000001,"15min":1.0840000000000001},"min":{"1min":0.78700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.423,"5min":2.5,"15min":2.5},"last":0.94499999999999995}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.1040000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1279999999999999,"15min":1.1459999999999999},"min":{"1min":0.879,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.575,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.026},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.139,"15min":1.181},"min":{"1min":0.68700000000000006,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.534,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.1279999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:02:10.474 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T13:02:10.769 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:10.769 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":173,"fsid":"c8f900da-3c0e-4eac-85d1-bc7dc62c8fc7","created":"2024-09-16T12:58:33.966359+0000","modified":"2024-09-16T13:02:03.831869+0000","last_up_change":"2024-09-16T13:01:33.961419+0000","last_in_change":"2024-09-16T12:59:39.968761+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T12:58:44.284177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T12:58:51.733915+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T12:59:00.193651+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T12:59:09.779825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":79,"snap_epoch":151,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"0d8a8277-07cd-40f1-8c98-98b8b51d1689","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6817","nonce":22186},{"type":"v1","addr":"172.21.15.42:6819","nonce":22186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6821","nonce":22186},{"type":"v1","addr":"172.21.15.42:6823","nonce":22186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":22186},{"type":"v1","addr":"172.21.15.42:6830","nonce":22186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6825","nonce":22186},{"type":"v1","addr":"172.21.15.42:6826","nonce":22186}]},"public_addr":"172.21.15.42:6819/22186","cluster_addr":"172.21.15.42:6823/22186","heartbeat_back_addr":"172.21.15.42:6830/22186","heartbeat_front_addr":"172.21.15.42:6826/22186","state":["exists","up"]},{"osd":1,"uuid":"6ce5996a-5e0e-4c9d-b268-27b34a14d830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":22184},{"type":"v1","addr":"172.21.15.42:6809","nonce":22184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":22184},{"type":"v1","addr":"172.21.15.42:6811","nonce":22184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":22184},{"type":"v1","addr":"172.21.15.42:6815","nonce":22184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":22184},{"type":"v1","addr":"172.21.15.42:6813","nonce":22184}]},"public_addr":"172.21.15.42:6809/22184","cluster_addr":"172.21.15.42:6811/22184","heartbeat_back_addr":"172.21.15.42:6815/22184","heartbeat_front_addr":"172.21.15.42:6813/22184","state":["exists","up"]},{"osd":2,"uuid":"156042ab-909e-4368-b510-84018cb4354f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":22185},{"type":"v1","addr":"172.21.15.42:6801","nonce":22185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":22185},{"type":"v1","addr":"172.21.15.42:6803","nonce":22185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":22185},{"type":"v1","addr":"172.21.15.42:6807","nonce":22185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":22185},{"type":"v1","addr":"172.21.15.42:6805","nonce":22185}]},"public_addr":"172.21.15.42:6801/22185","cluster_addr":"172.21.15.42:6803/22185","heartbeat_back_addr":"172.21.15.42:6807/22185","heartbeat_front_addr":"172.21.15.42:6805/22185","state":["exists","up"]},{"osd":3,"uuid":"b6c53823-446f-4413-a2ff-d2ebe8fe9c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":22187},{"type":"v1","addr":"172.21.15.42:6818","nonce":22187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":22187},{"type":"v1","addr":"172.21.15.42:6822","nonce":22187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":22187},{"type":"v1","addr":"172.21.15.42:6831","nonce":22187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":22187},{"type":"v1","addr":"172.21.15.42:6827","nonce":22187}]},"public_addr":"172.21.15.42:6818/22187","cluster_addr":"172.21.15.42:6822/22187","heartbeat_back_addr":"172.21.15.42:6831/22187","heartbeat_front_addr":"172.21.15.42:6827/22187","state":["exists","up"]},{"osd":4,"uuid":"278bfad1-010b-494e-ba61-bdf3935ca078","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":83,"up_from":108,"up_thru":147,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":33598},{"type":"v1","addr":"172.21.15.136:6817","nonce":33598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":33598},{"type":"v1","addr":"172.21.15.136:6819","nonce":33598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":33598},{"type":"v1","addr":"172.21.15.136:6823","nonce":33598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":33598},{"type":"v1","addr":"172.21.15.136:6821","nonce":33598}]},"public_addr":"172.21.15.136:6817/33598","cluster_addr":"172.21.15.136:6819/33598","heartbeat_back_addr":"172.21.15.136:6823/33598","heartbeat_front_addr":"172.21.15.136:6821/33598","state":["exists","up"]},{"osd":5,"uuid":"ed1af73b-b108-4189-8223-f017e634e3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22340},{"type":"v1","addr":"172.21.15.136:6809","nonce":22340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22340},{"type":"v1","addr":"172.21.15.136:6811","nonce":22340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22340},{"type":"v1","addr":"172.21.15.136:6815","nonce":22340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22340},{"type":"v1","addr":"172.21.15.136:6813","nonce":22340}]},"public_addr":"172.21.15.136:6809/22340","cluster_addr":"172.21.15.136:6811/22340","heartbeat_back_addr":"172.21.15.136:6815/22340","heartbeat_front_addr":"172.21.15.136:6813/22340","state":["exists","up"]},{"osd":6,"uuid":"ce904ba8-02e9-4579-9635-b4b6c6de1a08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":120,"up_from":147,"up_thru":147,"down_at":121,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":41077},{"type":"v1","addr":"172.21.15.136:6825","nonce":41077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":41077},{"type":"v1","addr":"172.21.15.136:6827","nonce":41077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":41077},{"type":"v1","addr":"172.21.15.136:6831","nonce":41077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":41077},{"type":"v1","addr":"172.21.15.136:6829","nonce":41077}]},"public_addr":"172.21.15.136:6825/41077","cluster_addr":"172.21.15.136:6827/41077","heartbeat_back_addr":"172.21.15.136:6831/41077","heartbeat_front_addr":"172.21.15.136:6829/41077","state":["exists","up"]},{"osd":7,"uuid":"d911da1b-2725-46a8-bbfa-8968de47271f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22341},{"type":"v1","addr":"172.21.15.136:6801","nonce":22341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22341},{"type":"v1","addr":"172.21.15.136:6803","nonce":22341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22341},{"type":"v1","addr":"172.21.15.136:6807","nonce":22341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22341},{"type":"v1","addr":"172.21.15.136:6805","nonce":22341}]},"public_addr":"172.21.15.136:6801/22341","cluster_addr":"172.21.15.136:6803/22341","heartbeat_back_addr":"172.21.15.136:6807/22341","heartbeat_front_addr":"172.21.15.136:6805/22341","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.264463+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.214474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.217927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.435822+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.451286+0000","dead_epoch":84},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.287403+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.440296+0000","dead_epoch":121},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T12:58:41.140630+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"4.12","osds":[5,1]}],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":1,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T13:02:11.783 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T13:02:11.783 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T13:02:11.939 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:02:11.939 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T13:02:11.940 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:02:11.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T13:02:12.258 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T13:02:12.268 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T13:02:12.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T13:02:12.407 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:02:12.407 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T13:02:12.407 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:02:12.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T13:02:12.723 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T13:02:12.736 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T13:02:12.736 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T13:02:12.887 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:02:12.887 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T13:02:12.887 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:02:12.900 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T13:02:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T13:02:13.225 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T13:02:13.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T13:02:13.376 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:02:13.376 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T13:02:13.376 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:02:13.392 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T13:02:13.700 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T13:02:13.714 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T13:02:13.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T13:02:13.860 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:02:13.861 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T13:02:13.861 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:02:13.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T13:02:14.183 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T13:02:14.193 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T13:02:14.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T13:02:14.339 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:02:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T13:02:14.339 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:02:14.351 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T13:02:14.651 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T13:02:14.662 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T13:02:14.662 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T13:02:14.806 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:02:14.806 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T13:02:14.806 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:02:14.819 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T13:02:15.116 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T13:02:15.126 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T13:02:15.126 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T13:02:15.277 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T13:02:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T13:02:15.277 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T13:02:15.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T13:02:15.590 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T13:02:15.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:02:15.900 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:15.904 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:02:15.916 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-16T13:02:14.292346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":1005,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21120,"num_write_kb":309526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19275,"ondisk_log_size":19275,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424296,"kb_used_data":16364,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752550424,"statfs":{"total":773094113280,"available":770611634176,"internally_reserved":0,"allocated":16756736,"data_stored":9594375,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.449957"},"pg_stats":[{"pgid":"4.1f","version":"151'339","reported_seq":4229,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865786+0000","last_change":"2024-09-16T13:01:38.032360+0000","last_active":"2024-09-16T13:02:02.865786+0000","last_peered":"2024-09-16T13:02:02.865786+0000","last_clean":"2024-09-16T13:02:02.865786+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:02:02.865786+0000","last_undegraded":"2024-09-16T13:02:02.865786+0000","last_fullsized":"2024-09-16T13:02:02.865786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:11.756487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015622850000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":333,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"150'514","reported_seq":17234,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856527+0000","last_change":"2024-09-16T13:01:38.062519+0000","last_active":"2024-09-16T13:02:02.856527+0000","last_peered":"2024-09-16T13:02:02.856527+0000","last_clean":"2024-09-16T13:02:02.856527+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:02:02.856527+0000","last_undegraded":"2024-09-16T13:02:02.856527+0000","last_fullsized":"2024-09-16T13:02:02.856527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:24.290894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"142'433","reported_seq":4984,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829498+0000","last_change":"2024-09-16T13:01:38.020672+0000","last_active":"2024-09-16T13:02:02.829498+0000","last_peered":"2024-09-16T13:02:02.829498+0000","last_clean":"2024-09-16T13:02:02.829498+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:02:02.829498+0000","last_undegraded":"2024-09-16T13:02:02.829498+0000","last_fullsized":"2024-09-16T13:02:02.829498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:04.274184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"151'1171","reported_seq":8013,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863870+0000","last_change":"2024-09-16T13:01:38.017984+0000","last_active":"2024-09-16T13:02:02.863870+0000","last_peered":"2024-09-16T13:02:02.863870+0000","last_clean":"2024-09-16T13:02:02.863870+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:02:02.863870+0000","last_undegraded":"2024-09-16T13:02:02.863870+0000","last_fullsized":"2024-09-16T13:02:02.863870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:57.444630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"150'493","reported_seq":22039,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827723+0000","last_change":"2024-09-16T13:01:38.026576+0000","last_active":"2024-09-16T13:02:02.827723+0000","last_peered":"2024-09-16T13:02:02.827723+0000","last_clean":"2024-09-16T13:02:02.827723+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:02:02.827723+0000","last_undegraded":"2024-09-16T13:02:02.827723+0000","last_fullsized":"2024-09-16T13:02:02.827723+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:04.244300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"150'685","reported_seq":7142,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828217+0000","last_change":"2024-09-16T13:01:38.026337+0000","last_active":"2024-09-16T13:02:02.828217+0000","last_peered":"2024-09-16T13:02:02.828217+0000","last_clean":"2024-09-16T13:02:02.828217+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:02:02.828217+0000","last_undegraded":"2024-09-16T13:02:02.828217+0000","last_fullsized":"2024-09-16T13:02:02.828217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:30.964357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"149'399","reported_seq":5474,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828856+0000","last_change":"2024-09-16T13:01:38.021608+0000","last_active":"2024-09-16T13:02:02.828856+0000","last_peered":"2024-09-16T13:02:02.828856+0000","last_clean":"2024-09-16T13:02:02.828856+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:02:02.828856+0000","last_undegraded":"2024-09-16T13:02:02.828856+0000","last_fullsized":"2024-09-16T13:02:02.828856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:26.702723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"150'440","reported_seq":6456,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827434+0000","last_change":"2024-09-16T13:01:38.024730+0000","last_active":"2024-09-16T13:02:02.827434+0000","last_peered":"2024-09-16T13:02:02.827434+0000","last_clean":"2024-09-16T13:02:02.827434+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:02:02.827434+0000","last_undegraded":"2024-09-16T13:02:02.827434+0000","last_fullsized":"2024-09-16T13:02:02.827434+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:23.687215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"150'889","reported_seq":7631,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829608+0000","last_change":"2024-09-16T13:01:38.021503+0000","last_active":"2024-09-16T13:02:02.829608+0000","last_peered":"2024-09-16T13:02:02.829608+0000","last_clean":"2024-09-16T13:02:02.829608+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:02:02.829608+0000","last_undegraded":"2024-09-16T13:02:02.829608+0000","last_fullsized":"2024-09-16T13:02:02.829608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:28.627072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"150'508","reported_seq":3310,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856528+0000","last_change":"2024-09-16T13:01:38.062149+0000","last_active":"2024-09-16T13:02:02.856528+0000","last_peered":"2024-09-16T13:02:02.856528+0000","last_clean":"2024-09-16T13:02:02.856528+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:02:02.856528+0000","last_undegraded":"2024-09-16T13:02:02.856528+0000","last_fullsized":"2024-09-16T13:02:02.856528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:18.285140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"150'523","reported_seq":16214,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829042+0000","last_change":"2024-09-16T13:01:38.021224+0000","last_active":"2024-09-16T13:02:02.829042+0000","last_peered":"2024-09-16T13:02:02.829042+0000","last_clean":"2024-09-16T13:02:02.829042+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:02:02.829042+0000","last_undegraded":"2024-09-16T13:02:02.829042+0000","last_fullsized":"2024-09-16T13:02:02.829042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:59.380538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"149'579","reported_seq":23227,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829478+0000","last_change":"2024-09-16T13:01:38.020686+0000","last_active":"2024-09-16T13:02:02.829478+0000","last_peered":"2024-09-16T13:02:02.829478+0000","last_clean":"2024-09-16T13:02:02.829478+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:02:02.829478+0000","last_undegraded":"2024-09-16T13:02:02.829478+0000","last_fullsized":"2024-09-16T13:02:02.829478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.084802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"151'512","reported_seq":6232,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827930+0000","last_change":"2024-09-16T13:01:38.027017+0000","last_active":"2024-09-16T13:02:02.827930+0000","last_peered":"2024-09-16T13:02:02.827930+0000","last_clean":"2024-09-16T13:02:02.827930+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:02:02.827930+0000","last_undegraded":"2024-09-16T13:02:02.827930+0000","last_fullsized":"2024-09-16T13:02:02.827930+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:56.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"150'371","reported_seq":3123,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828778+0000","last_change":"2024-09-16T13:01:38.019785+0000","last_active":"2024-09-16T13:02:02.828778+0000","last_peered":"2024-09-16T13:02:02.828778+0000","last_clean":"2024-09-16T13:02:02.828778+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:02:02.828778+0000","last_undegraded":"2024-09-16T13:02:02.828778+0000","last_fullsized":"2024-09-16T13:02:02.828778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:40.089652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"149'363","reported_seq":18827,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828605+0000","last_change":"2024-09-16T13:01:38.019836+0000","last_active":"2024-09-16T13:02:02.828605+0000","last_peered":"2024-09-16T13:02:02.828605+0000","last_clean":"2024-09-16T13:02:02.828605+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:02:02.828605+0000","last_undegraded":"2024-09-16T13:02:02.828605+0000","last_fullsized":"2024-09-16T13:02:02.828605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:02.420719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"139'335","reported_seq":5458,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.018514+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:57.740196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865667+0000","last_change":"2024-09-16T13:01:35.025514+0000","last_active":"2024-09-16T13:02:02.865667+0000","last_peered":"2024-09-16T13:02:02.865667+0000","last_clean":"2024-09-16T13:02:02.865667+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:02:02.865667+0000","last_undegraded":"2024-09-16T13:02:02.865667+0000","last_fullsized":"2024-09-16T13:02:02.865667+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:27.902219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"146'451","reported_seq":3775,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828186+0000","last_change":"2024-09-16T13:01:38.027021+0000","last_active":"2024-09-16T13:02:02.828186+0000","last_peered":"2024-09-16T13:02:02.828186+0000","last_clean":"2024-09-16T13:02:02.828186+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:02:02.828186+0000","last_undegraded":"2024-09-16T13:02:02.828186+0000","last_fullsized":"2024-09-16T13:02:02.828186+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:41.831955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.864188+0000","last_change":"2024-09-16T13:01:34.980687+0000","last_active":"2024-09-16T13:02:02.864188+0000","last_peered":"2024-09-16T13:02:02.864188+0000","last_clean":"2024-09-16T13:02:02.864188+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:02:02.864188+0000","last_undegraded":"2024-09-16T13:02:02.864188+0000","last_fullsized":"2024-09-16T13:02:02.864188+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:44.616572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"149'690","reported_seq":7327,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828561+0000","last_change":"2024-09-16T13:01:38.020063+0000","last_active":"2024-09-16T13:02:02.828561+0000","last_peered":"2024-09-16T13:02:02.828561+0000","last_clean":"2024-09-16T13:02:02.828561+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:02:02.828561+0000","last_undegraded":"2024-09-16T13:02:02.828561+0000","last_fullsized":"2024-09-16T13:02:02.828561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:14.651946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":161,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865756+0000","last_change":"2024-09-16T12:58:56.031957+0000","last_active":"2024-09-16T13:02:02.865756+0000","last_peered":"2024-09-16T13:02:02.865756+0000","last_clean":"2024-09-16T13:02:02.865756+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:02:02.865756+0000","last_undegraded":"2024-09-16T13:02:02.865756+0000","last_fullsized":"2024-09-16T13:02:02.865756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.058322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"151'687","reported_seq":3219,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856470+0000","last_change":"2024-09-16T13:01:38.061542+0000","last_active":"2024-09-16T13:02:02.856470+0000","last_peered":"2024-09-16T13:02:02.856470+0000","last_clean":"2024-09-16T13:02:02.856470+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:02:02.856470+0000","last_undegraded":"2024-09-16T13:02:02.856470+0000","last_fullsized":"2024-09-16T13:02:02.856470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:04.847040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":122,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.864151+0000","last_change":"2024-09-16T12:59:40.994481+0000","last_active":"2024-09-16T13:02:02.864151+0000","last_peered":"2024-09-16T13:02:02.864151+0000","last_clean":"2024-09-16T13:02:02.864151+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:02:02.864151+0000","last_undegraded":"2024-09-16T13:02:02.864151+0000","last_fullsized":"2024-09-16T13:02:02.864151+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:19.889048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"150'586","reported_seq":17110,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.864374+0000","last_change":"2024-09-16T13:01:38.114806+0000","last_active":"2024-09-16T13:02:02.864374+0000","last_peered":"2024-09-16T13:02:02.864374+0000","last_clean":"2024-09-16T13:02:02.864374+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:02:02.864374+0000","last_undegraded":"2024-09-16T13:02:02.864374+0000","last_fullsized":"2024-09-16T13:02:02.864374+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:00.605679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":124,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827707+0000","last_change":"2024-09-16T13:00:56.529588+0000","last_active":"2024-09-16T13:02:02.827707+0000","last_peered":"2024-09-16T13:02:02.827707+0000","last_clean":"2024-09-16T13:02:02.827707+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:02:02.827707+0000","last_undegraded":"2024-09-16T13:02:02.827707+0000","last_fullsized":"2024-09-16T13:02:02.827707+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.120921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":191,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829038+0000","last_change":"2024-09-16T12:59:41.095911+0000","last_active":"2024-09-16T13:02:02.829038+0000","last_peered":"2024-09-16T13:02:02.829038+0000","last_clean":"2024-09-16T13:02:02.829038+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:02:02.829038+0000","last_undegraded":"2024-09-16T13:02:02.829038+0000","last_fullsized":"2024-09-16T13:02:02.829038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:20.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"148'501","reported_seq":17937,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865403+0000","last_change":"2024-09-16T13:01:38.031901+0000","last_active":"2024-09-16T13:02:02.865403+0000","last_peered":"2024-09-16T13:02:02.865403+0000","last_clean":"2024-09-16T13:02:02.865403+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:02:02.865403+0000","last_undegraded":"2024-09-16T13:02:02.865403+0000","last_fullsized":"2024-09-16T13:02:02.865403+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:27.630481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411569999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":122,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863991+0000","last_change":"2024-09-16T12:59:40.991405+0000","last_active":"2024-09-16T13:02:02.863991+0000","last_peered":"2024-09-16T13:02:02.863991+0000","last_clean":"2024-09-16T13:02:02.863991+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:02:02.863991+0000","last_undegraded":"2024-09-16T13:02:02.863991+0000","last_fullsized":"2024-09-16T13:02:02.863991+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:40.877096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.863815+0000","last_change":"2024-09-16T12:58:55.646773+0000","last_active":"2024-09-16T13:02:02.863815+0000","last_peered":"2024-09-16T13:02:02.863815+0000","last_clean":"2024-09-16T13:02:02.863815+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:02:02.863815+0000","last_undegraded":"2024-09-16T13:02:02.863815+0000","last_fullsized":"2024-09-16T13:02:02.863815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:14.695105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"149'407","reported_seq":2775,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.864304+0000","last_change":"2024-09-16T13:01:38.113780+0000","last_active":"2024-09-16T13:02:02.864304+0000","last_peered":"2024-09-16T13:02:02.864304+0000","last_clean":"2024-09-16T13:02:02.864304+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:02:02.864304+0000","last_undegraded":"2024-09-16T13:02:02.864304+0000","last_fullsized":"2024-09-16T13:02:02.864304+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.723284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":169,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856355+0000","last_change":"2024-09-16T13:00:56.528658+0000","last_active":"2024-09-16T13:02:02.856355+0000","last_peered":"2024-09-16T13:02:02.856355+0000","last_clean":"2024-09-16T13:02:02.856355+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:02:02.856355+0000","last_undegraded":"2024-09-16T13:02:02.856355+0000","last_fullsized":"2024-09-16T13:02:02.856355+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:05.220262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":179,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865704+0000","last_change":"2024-09-16T12:59:40.998432+0000","last_active":"2024-09-16T13:02:02.865704+0000","last_peered":"2024-09-16T13:02:02.865704+0000","last_clean":"2024-09-16T13:02:02.865704+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:02:02.865704+0000","last_undegraded":"2024-09-16T13:02:02.865704+0000","last_fullsized":"2024-09-16T13:02:02.865704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:44.180033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"151'434","reported_seq":7322,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856609+0000","last_change":"2024-09-16T13:01:38.062717+0000","last_active":"2024-09-16T13:02:02.856609+0000","last_peered":"2024-09-16T13:02:02.856609+0000","last_clean":"2024-09-16T13:02:02.856609+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:02:02.856609+0000","last_undegraded":"2024-09-16T13:02:02.856609+0000","last_fullsized":"2024-09-16T13:02:02.856609+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:55.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"151'4","reported_seq":158,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829055+0000","last_change":"2024-09-16T12:59:02.146282+0000","last_active":"2024-09-16T13:02:02.829055+0000","last_peered":"2024-09-16T13:02:02.829055+0000","last_clean":"2024-09-16T13:02:02.829055+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:02:02.829055+0000","last_undegraded":"2024-09-16T13:02:02.829055+0000","last_fullsized":"2024-09-16T13:02:02.829055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:00.662491+0000","last_clean_scrub_stamp":"2024-09-16T12:59:00.662491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:06.771159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":164,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.829054+0000","last_change":"2024-09-16T12:58:55.652492+0000","last_active":"2024-09-16T13:02:02.829054+0000","last_peered":"2024-09-16T13:02:02.829054+0000","last_clean":"2024-09-16T13:02:02.829054+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:02:02.829054+0000","last_undegraded":"2024-09-16T13:02:02.829054+0000","last_fullsized":"2024-09-16T13:02:02.829054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:22.232695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":249,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865907+0000","last_change":"2024-09-16T12:58:46.133737+0000","last_active":"2024-09-16T13:02:02.865907+0000","last_peered":"2024-09-16T13:02:02.865907+0000","last_clean":"2024-09-16T13:02:02.865907+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:02:02.865907+0000","last_undegraded":"2024-09-16T13:02:02.865907+0000","last_fullsized":"2024-09-16T13:02:02.865907+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:44.530413+0000","last_clean_scrub_stamp":"2024-09-16T12:58:44.530413+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:05.647543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"150'444","reported_seq":3720,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865541+0000","last_change":"2024-09-16T13:01:38.031044+0000","last_active":"2024-09-16T13:02:02.865541+0000","last_peered":"2024-09-16T13:02:02.865541+0000","last_clean":"2024-09-16T13:02:02.865541+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:02:02.865541+0000","last_undegraded":"2024-09-16T13:02:02.865541+0000","last_fullsized":"2024-09-16T13:02:02.865541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:16.451880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602565,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"150'977","reported_seq":18782,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.828748+0000","last_change":"2024-09-16T13:01:38.019797+0000","last_active":"2024-09-16T13:02:02.828748+0000","last_peered":"2024-09-16T13:02:02.828748+0000","last_clean":"2024-09-16T13:02:02.828748+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:02:02.828748+0000","last_undegraded":"2024-09-16T13:02:02.828748+0000","last_fullsized":"2024-09-16T13:02:02.828748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:25.548245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"149'328","reported_seq":847,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827607+0000","last_change":"2024-09-16T13:01:38.024649+0000","last_active":"2024-09-16T13:02:02.827607+0000","last_peered":"2024-09-16T13:02:02.827607+0000","last_clean":"2024-09-16T13:02:02.827607+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:02:02.827607+0000","last_undegraded":"2024-09-16T13:02:02.827607+0000","last_fullsized":"2024-09-16T13:02:02.827607+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:31.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"150'1380","reported_seq":6972,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.865373+0000","last_change":"2024-09-16T13:01:38.031686+0000","last_active":"2024-09-16T13:02:02.865373+0000","last_peered":"2024-09-16T13:02:02.865373+0000","last_clean":"2024-09-16T13:02:02.865373+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:02:02.865373+0000","last_undegraded":"2024-09-16T13:02:02.865373+0000","last_fullsized":"2024-09-16T13:02:02.865373+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:00.052447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093887699999999996,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"144'974","reported_seq":6637,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.856424+0000","last_change":"2024-09-16T13:01:38.060958+0000","last_active":"2024-09-16T13:02:02.856424+0000","last_peered":"2024-09-16T13:02:02.856424+0000","last_clean":"2024-09-16T13:02:02.856424+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:02:02.856424+0000","last_undegraded":"2024-09-16T13:02:02.856424+0000","last_fullsized":"2024-09-16T13:02:02.856424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:00.508887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"151'553","reported_seq":5350,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827827+0000","last_change":"2024-09-16T13:01:38.025883+0000","last_active":"2024-09-16T13:02:02.827827+0000","last_peered":"2024-09-16T13:02:02.827827+0000","last_clean":"2024-09-16T13:02:02.827827+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:02:02.827827+0000","last_undegraded":"2024-09-16T13:02:02.827827+0000","last_fullsized":"2024-09-16T13:02:02.827827+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:54.360124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"150'434","reported_seq":4405,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827901+0000","last_change":"2024-09-16T13:01:38.027791+0000","last_active":"2024-09-16T13:02:02.827901+0000","last_peered":"2024-09-16T13:02:02.827901+0000","last_clean":"2024-09-16T13:02:02.827901+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:02:02.827901+0000","last_undegraded":"2024-09-16T13:02:02.827901+0000","last_fullsized":"2024-09-16T13:02:02.827901+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:02.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"150'876","reported_seq":4051,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.827980+0000","last_change":"2024-09-16T13:01:38.025990+0000","last_active":"2024-09-16T13:02:02.827980+0000","last_peered":"2024-09-16T13:02:02.827980+0000","last_clean":"2024-09-16T13:02:02.827980+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:02:02.827980+0000","last_undegraded":"2024-09-16T13:02:02.827980+0000","last_fullsized":"2024-09-16T13:02:02.827980+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:12.558678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"149'919","reported_seq":36749,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-16T13:02:02.864270+0000","last_change":"2024-09-16T13:01:38.113695+0000","last_active":"2024-09-16T13:02:02.864270+0000","last_peered":"2024-09-16T13:02:02.864270+0000","last_clean":"2024-09-16T13:02:02.864270+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:02:02.864270+0000","last_undegraded":"2024-09-16T13:02:02.864270+0000","last_fullsized":"2024-09-16T13:02:02.864270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.609361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21015,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":19195,"ondisk_log_size":19195,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192522,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301756,"kb_used_data":2028,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070084,"statfs":{"total":96636764160,"available":96327766016,"internally_reserved":0,"allocated":2076672,"data_stored":1051715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303568,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068272,"statfs":{"total":96636764160,"available":96325910528,"internally_reserved":0,"allocated":2088960,"data_stored":1051902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.85599999999999998},"min":{"1min":0.48099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1499999999999999,"5min":1.272,"15min":1.272},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.83999999999999997},"min":{"1min":0.55100000000000005,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.391,"15min":1.391},"last":0.57899999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.97299999999999998,"15min":0.89200000000000002},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.333,"5min":14.275,"15min":14.275},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.58699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3600000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.752}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.58099999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2430000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.996,"5min":1.071,"15min":1.103},"min":{"1min":0.61799999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4830000000000001,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.95299999999999996}]},{"osd":3,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.071,"15min":1.0669999999999999},"min":{"1min":0.61899999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.2989999999999999,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.115},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6299999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.272}]},{"osd":4,"last update":"Mon Sep 16 13:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0109999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":1.446},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.077,"15min":1.137},"min":{"1min":0.65300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4379999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.29},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1850000000000001,"15min":1.2450000000000001},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.673,"5min":1.748,"15min":1.748},"last":1.369}]}]},{"osd":4,"up_from":108,"seq":463856467985,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309436,"kb_used_data":2032,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062404,"statfs":{"total":96636764160,"available":96319901696,"internally_reserved":0,"allocated":2080768,"data_stored":1051734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.83799999999999997}]},{"osd":1,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.92100000000000004}]},{"osd":2,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.80700000000000005}]},{"osd":5,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":7,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.876},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.97299999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299396,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072444,"statfs":{"total":96636764160,"available":96330182656,"internally_reserved":0,"allocated":2088960,"data_stored":1051902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.995,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71799999999999997,"15min":0.66700000000000004},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0940000000000001,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.749}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.232,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.51500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.103,"5min":1.528,"15min":1.528},"last":0.83199999999999996}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98399999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.63},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.49299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5740000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.0940000000000001}]},{"osd":5,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.109,"15min":1.129},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8520000000000001,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":1.292},{"interface":"front","average":{"1min":1.171,"5min":1.1060000000000001,"15min":1.1180000000000001},"min":{"1min":0.56000000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.1659999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.1890000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1299999999999999},"min":{"1min":0.61099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.054},{"interface":"front","average":{"1min":1.167,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.77700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9830000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.2649999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299388,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072452,"statfs":{"total":96636764160,"available":96330190848,"internally_reserved":0,"allocated":2072576,"data_stored":1051222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.314,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.0980000000000001,"5min":10.654,"15min":10.654},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.77100000000000002},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.97899999999999998,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.66900000000000004}]},{"osd":1,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.52200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99099999999999999,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.499}]},{"osd":3,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.89200000000000002,"15min":0.80400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.2430000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.077,"5min":1.647,"15min":1.647},"last":0.59599999999999997}]},{"osd":4,"last update":"Mon Sep 16 13:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.85999999999999999}]},{"osd":5,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.093,"15min":1.1930000000000001},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.226,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.874},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0740000000000001,"15min":1.1439999999999999},"min":{"1min":0.60799999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.383,"5min":2.125,"15min":2.125},"last":0.89200000000000002}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.749},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.139,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.478,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.054,"5min":1.1539999999999999,"15min":1.2450000000000001},"min":{"1min":0.63700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.538,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.65400000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303600,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068240,"statfs":{"total":96636764160,"available":96325877760,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.88600000000000001,"15min":0.84199999999999997},"min":{"1min":0.39100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.1899999999999999,"5min":9.407,"15min":9.407},"last":0.627},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72499999999999998}]},{"osd":2,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.85999999999999999,"15min":0.94599999999999995},"min":{"1min":0.46899999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.319,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.754},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0469999999999999,"5min":2.254,"15min":2.254},"last":0.54300000000000004}]},{"osd":3,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.36399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83199999999999996,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":1.028},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4750000000000001,"5min":2.044,"15min":2.044},"last":0.65600000000000003}]},{"osd":4,"last update":"Mon Sep 16 13:01:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.80200000000000005}]},{"osd":5,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0149999999999999,"15min":1.0640000000000001},"min":{"1min":0.56399999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4730000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.999,"5min":1.133,"15min":1.2050000000000001},"min":{"1min":0.60999999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.325,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.0169999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.1899999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.129,"15min":1.177},"min":{"1min":0.745,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.327,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1599999999999999},"min":{"1min":0.70099999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.427,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.90100000000000002}]}]},{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303600,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068240,"statfs":{"total":96636764160,"available":96325877760,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5920000000000001,"5min":1.73,"15min":1.73},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.145,"15min":1.26},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3080000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.69999999999999996}]},{"osd":1,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0249999999999999,"15min":1.1559999999999999},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.208,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.56499999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5149999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.85099999999999998}]},{"osd":2,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0569999999999999},"min":{"1min":0.55400000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.298,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.143,"15min":1.2030000000000001},"min":{"1min":0.50900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1839999999999999,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.131}]},{"osd":3,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0109999999999999,"15min":1.0720000000000001},"min":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.984,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.2010000000000001,"15min":1.298},"min":{"1min":0.66900000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9299999999999999,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.032}]},{"osd":4,"last update":"Mon Sep 16 13:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.1919999999999999}]},{"osd":5,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.1759999999999999,"15min":1.3029999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.135,"5min":6.282,"15min":6.282},"last":1.103},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.93600000000000005,"15min":0.97799999999999998},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.2549999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.96199999999999997}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.218}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303552,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068288,"statfs":{"total":96636764160,"available":96325926912,"internally_reserved":0,"allocated":2080768,"data_stored":1051734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.73999999999999999},"min":{"1min":0.38100000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.97799999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.254,"15min":0.254},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.75900000000000001}]},{"osd":2,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72799999999999998,"15min":0.77600000000000002},"min":{"1min":0.35699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.90400000000000003,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.181,"5min":3.024,"15min":3.024},"last":0.66600000000000004}]},{"osd":3,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.95999999999999996,"15min":1.034},"min":{"1min":0.53600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2250000000000001,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.49199999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.151,"5min":3.544,"15min":3.544},"last":0.91900000000000004}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.1879999999999999}]},{"osd":5,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.25,"15min":1.3979999999999999},"min":{"1min":0.55700000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.619,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.997},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0600000000000001,"15min":1.0840000000000001},"min":{"1min":0.78700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.423,"5min":2.5,"15min":2.5},"last":0.94499999999999995}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.1040000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1279999999999999,"15min":1.1459999999999999},"min":{"1min":0.879,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.575,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.026},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.139,"15min":1.181},"min":{"1min":0.68700000000000006,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.534,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.1279999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:02:15.921 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.921 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.921 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.922 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.922 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.922 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.922 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.922 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.922 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.922 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.922 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.922 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.923 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.923 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.923 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.923 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.923 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T12:58:52.605804+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=58, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.923 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.923 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T12:58:52.605804+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=58, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.923 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.923 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T12:58:52.605804+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=58, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.924 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.924 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T12:58:52.605804+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=58, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.924 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.924 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-16T12:59:00.662491+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=0, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.924 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T12:58:52.605804+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=58, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.924 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.924 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-16T12:59:00.662491+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=0, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.924 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T12:58:52.605804+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=58, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.924 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.925 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-16T12:59:00.662491+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=0, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.925 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T12:58:52.605804+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=58, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.925 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.925 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-16T12:59:00.662491+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=0, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.925 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T12:58:52.605804+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=58, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.925 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T12:58:44.530413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=58, tm_sec=44, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.925 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.925 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.925 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.926 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.926 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.926 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.926 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.926 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.926 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:15.926 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T13:02:35.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:02:36.239 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:36.242 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:02:36.255 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-16T13:02:34.298898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":1005,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21120,"num_write_kb":309526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19939,"ondisk_log_size":19939,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424236,"kb_used_data":16364,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752550484,"statfs":{"total":773094113280,"available":770611695616,"internally_reserved":0,"allocated":16756736,"data_stored":9594375,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003947"},"pg_stats":[{"pgid":"4.1f","version":"173'363","reported_seq":4264,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:21.702577+0000","last_change":"2024-09-16T13:02:21.702540+0000","last_active":"2024-09-16T13:02:21.702577+0000","last_peered":"2024-09-16T13:02:21.702577+0000","last_clean":"2024-09-16T13:02:21.702577+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:02:21.702577+0000","last_undegraded":"2024-09-16T13:02:21.702577+0000","last_fullsized":"2024-09-16T13:02:21.702577+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'363","last_scrub_stamp":"2024-09-16T13:02:21.702427+0000","last_deep_scrub":"173'363","last_deep_scrub_stamp":"2024-09-16T13:02:21.702427+0000","last_clean_scrub_stamp":"2024-09-16T13:02:21.702427+0000","objects_scrubbed":24,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:15.774917+0000","scrub_duration":0.024839829000000001,"objects_trimmed":0,"snaptrim_duration":0.0015622850000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":333,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"150'514","reported_seq":17235,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:03.845154+0000","last_change":"2024-09-16T13:01:38.062519+0000","last_active":"2024-09-16T13:02:03.845154+0000","last_peered":"2024-09-16T13:02:03.845154+0000","last_clean":"2024-09-16T13:02:03.845154+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:02:03.845154+0000","last_undegraded":"2024-09-16T13:02:03.845154+0000","last_fullsized":"2024-09-16T13:02:03.845154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"173'453","reported_seq":5015,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:19.621025+0000","last_change":"2024-09-16T13:02:19.620977+0000","last_active":"2024-09-16T13:02:19.621025+0000","last_peered":"2024-09-16T13:02:19.621025+0000","last_clean":"2024-09-16T13:02:19.621025+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:02:19.621025+0000","last_undegraded":"2024-09-16T13:02:19.621025+0000","last_fullsized":"2024-09-16T13:02:19.621025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'453","last_scrub_stamp":"2024-09-16T13:02:19.620828+0000","last_deep_scrub":"173'453","last_deep_scrub_stamp":"2024-09-16T13:02:19.620828+0000","last_clean_scrub_stamp":"2024-09-16T13:02:19.620828+0000","objects_scrubbed":20,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:00.490290+0000","scrub_duration":0.027059613999999999,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"173'1210","reported_seq":8064,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:16.511218+0000","last_change":"2024-09-16T13:02:16.511154+0000","last_active":"2024-09-16T13:02:16.511218+0000","last_peered":"2024-09-16T13:02:16.511218+0000","last_clean":"2024-09-16T13:02:16.511218+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:02:16.511218+0000","last_undegraded":"2024-09-16T13:02:16.511218+0000","last_fullsized":"2024-09-16T13:02:16.511218+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'1210","last_scrub_stamp":"2024-09-16T13:02:16.510938+0000","last_deep_scrub":"173'1210","last_deep_scrub_stamp":"2024-09-16T13:02:16.510938+0000","last_clean_scrub_stamp":"2024-09-16T13:02:16.510938+0000","objects_scrubbed":39,"log_size":1210,"log_dups_size":0,"ondisk_log_size":1210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:09.257774+0000","scrub_duration":0.059094725000000001,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"150'493","reported_seq":22040,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:05.167348+0000","last_change":"2024-09-16T13:01:38.026576+0000","last_active":"2024-09-16T13:02:05.167348+0000","last_peered":"2024-09-16T13:02:05.167348+0000","last_clean":"2024-09-16T13:02:05.167348+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:02:05.167348+0000","last_undegraded":"2024-09-16T13:02:05.167348+0000","last_fullsized":"2024-09-16T13:02:05.167348+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"150'685","reported_seq":7143,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:05.166966+0000","last_change":"2024-09-16T13:01:38.026337+0000","last_active":"2024-09-16T13:02:05.166966+0000","last_peered":"2024-09-16T13:02:05.166966+0000","last_clean":"2024-09-16T13:02:05.166966+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:02:05.166966+0000","last_undegraded":"2024-09-16T13:02:05.166966+0000","last_fullsized":"2024-09-16T13:02:05.166966+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"173'427","reported_seq":5514,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:24.649076+0000","last_change":"2024-09-16T13:02:24.649040+0000","last_active":"2024-09-16T13:02:24.649076+0000","last_peered":"2024-09-16T13:02:24.649076+0000","last_clean":"2024-09-16T13:02:24.649076+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:02:24.649076+0000","last_undegraded":"2024-09-16T13:02:24.649076+0000","last_fullsized":"2024-09-16T13:02:24.649076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'427","last_scrub_stamp":"2024-09-16T13:02:24.648909+0000","last_deep_scrub":"173'427","last_deep_scrub_stamp":"2024-09-16T13:02:24.648909+0000","last_clean_scrub_stamp":"2024-09-16T13:02:24.648909+0000","objects_scrubbed":28,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:23.167553+0000","scrub_duration":0.034920038,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"173'468","reported_seq":6496,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:18.944855+0000","last_change":"2024-09-16T13:02:18.944778+0000","last_active":"2024-09-16T13:02:18.944855+0000","last_peered":"2024-09-16T13:02:18.944855+0000","last_clean":"2024-09-16T13:02:18.944855+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:02:18.944855+0000","last_undegraded":"2024-09-16T13:02:18.944855+0000","last_fullsized":"2024-09-16T13:02:18.944855+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'468","last_scrub_stamp":"2024-09-16T13:02:18.944663+0000","last_deep_scrub":"173'468","last_deep_scrub_stamp":"2024-09-16T13:02:18.944663+0000","last_clean_scrub_stamp":"2024-09-16T13:02:18.944663+0000","objects_scrubbed":28,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:07.109499+0000","scrub_duration":0.035887090000000003,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"173'925","reported_seq":7679,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:21.689278+0000","last_change":"2024-09-16T13:02:21.689215+0000","last_active":"2024-09-16T13:02:21.689278+0000","last_peered":"2024-09-16T13:02:21.689278+0000","last_clean":"2024-09-16T13:02:21.689278+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:02:21.689278+0000","last_undegraded":"2024-09-16T13:02:21.689278+0000","last_fullsized":"2024-09-16T13:02:21.689278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'925","last_scrub_stamp":"2024-09-16T13:02:21.689005+0000","last_deep_scrub":"173'925","last_deep_scrub_stamp":"2024-09-16T13:02:21.689005+0000","last_clean_scrub_stamp":"2024-09-16T13:02:21.689005+0000","objects_scrubbed":36,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:21.331466+0000","scrub_duration":0.053280920000000002,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"150'508","reported_seq":3311,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:03.845220+0000","last_change":"2024-09-16T13:01:38.062149+0000","last_active":"2024-09-16T13:02:03.845220+0000","last_peered":"2024-09-16T13:02:03.845220+0000","last_clean":"2024-09-16T13:02:03.845220+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:02:03.845220+0000","last_undegraded":"2024-09-16T13:02:03.845220+0000","last_fullsized":"2024-09-16T13:02:03.845220+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"173'555","reported_seq":16258,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:23.633663+0000","last_change":"2024-09-16T13:02:23.633623+0000","last_active":"2024-09-16T13:02:23.633663+0000","last_peered":"2024-09-16T13:02:23.633663+0000","last_clean":"2024-09-16T13:02:23.633663+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:02:23.633663+0000","last_undegraded":"2024-09-16T13:02:23.633663+0000","last_fullsized":"2024-09-16T13:02:23.633663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'555","last_scrub_stamp":"2024-09-16T13:02:23.633505+0000","last_deep_scrub":"173'555","last_deep_scrub_stamp":"2024-09-16T13:02:23.633505+0000","last_clean_scrub_stamp":"2024-09-16T13:02:23.633505+0000","objects_scrubbed":32,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:25.486925+0000","scrub_duration":0.037584139000000003,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"173'607","reported_seq":23267,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:22.622952+0000","last_change":"2024-09-16T13:02:22.622911+0000","last_active":"2024-09-16T13:02:22.622952+0000","last_peered":"2024-09-16T13:02:22.622952+0000","last_clean":"2024-09-16T13:02:22.622952+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:02:22.622952+0000","last_undegraded":"2024-09-16T13:02:22.622952+0000","last_fullsized":"2024-09-16T13:02:22.622952+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'607","last_scrub_stamp":"2024-09-16T13:02:22.622800+0000","last_deep_scrub":"173'607","last_deep_scrub_stamp":"2024-09-16T13:02:22.622800+0000","last_clean_scrub_stamp":"2024-09-16T13:02:22.622800+0000","objects_scrubbed":28,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:07:41.667158+0000","scrub_duration":0.035254847999999998,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"151'512","reported_seq":6233,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:05.167465+0000","last_change":"2024-09-16T13:01:38.027017+0000","last_active":"2024-09-16T13:02:05.167465+0000","last_peered":"2024-09-16T13:02:05.167465+0000","last_clean":"2024-09-16T13:02:05.167465+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:02:05.167465+0000","last_undegraded":"2024-09-16T13:02:05.167465+0000","last_fullsized":"2024-09-16T13:02:05.167465+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"150'371","reported_seq":3124,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:03.845608+0000","last_change":"2024-09-16T13:01:38.019785+0000","last_active":"2024-09-16T13:02:03.845608+0000","last_peered":"2024-09-16T13:02:03.845608+0000","last_clean":"2024-09-16T13:02:03.845608+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:02:03.845608+0000","last_undegraded":"2024-09-16T13:02:03.845608+0000","last_fullsized":"2024-09-16T13:02:03.845608+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"173'399","reported_seq":18875,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:29.622369+0000","last_change":"2024-09-16T13:02:29.622322+0000","last_active":"2024-09-16T13:02:29.622369+0000","last_peered":"2024-09-16T13:02:29.622369+0000","last_clean":"2024-09-16T13:02:29.622369+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:02:29.622369+0000","last_undegraded":"2024-09-16T13:02:29.622369+0000","last_fullsized":"2024-09-16T13:02:29.622369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'399","last_scrub_stamp":"2024-09-16T13:02:29.622174+0000","last_deep_scrub":"173'399","last_deep_scrub_stamp":"2024-09-16T13:02:29.622174+0000","last_clean_scrub_stamp":"2024-09-16T13:02:29.622174+0000","objects_scrubbed":36,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:35.334013+0000","scrub_duration":0.046116155999999998,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"139'335","reported_seq":5459,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:05.164645+0000","last_change":"2024-09-16T13:01:38.018514+0000","last_active":"2024-09-16T13:02:05.164645+0000","last_peered":"2024-09-16T13:02:05.164645+0000","last_clean":"2024-09-16T13:02:05.164645+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:02:05.164645+0000","last_undegraded":"2024-09-16T13:02:05.164645+0000","last_fullsized":"2024-09-16T13:02:05.164645+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:22.662212+0000","last_change":"2024-09-16T13:02:22.662137+0000","last_active":"2024-09-16T13:02:22.662212+0000","last_peered":"2024-09-16T13:02:22.662212+0000","last_clean":"2024-09-16T13:02:22.662212+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:02:22.662212+0000","last_undegraded":"2024-09-16T13:02:22.662212+0000","last_fullsized":"2024-09-16T13:02:22.662212+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:22.661894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:22.661894+0000","last_clean_scrub_stamp":"2024-09-16T13:02:22.661894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:38:51.676316+0000","scrub_duration":0.003126827,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"173'481","reported_seq":3817,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:28.977934+0000","last_change":"2024-09-16T13:02:28.977888+0000","last_active":"2024-09-16T13:02:28.977934+0000","last_peered":"2024-09-16T13:02:28.977934+0000","last_clean":"2024-09-16T13:02:28.977934+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:02:28.977934+0000","last_undegraded":"2024-09-16T13:02:28.977934+0000","last_fullsized":"2024-09-16T13:02:28.977934+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'481","last_scrub_stamp":"2024-09-16T13:02:28.977762+0000","last_deep_scrub":"173'481","last_deep_scrub_stamp":"2024-09-16T13:02:28.977762+0000","last_clean_scrub_stamp":"2024-09-16T13:02:28.977762+0000","objects_scrubbed":30,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:41.014327+0000","scrub_duration":0.031695378000000003,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:05.167938+0000","last_change":"2024-09-16T13:01:34.980687+0000","last_active":"2024-09-16T13:02:05.167938+0000","last_peered":"2024-09-16T13:02:05.167938+0000","last_clean":"2024-09-16T13:02:05.167938+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:02:05.167938+0000","last_undegraded":"2024-09-16T13:02:05.167938+0000","last_fullsized":"2024-09-16T13:02:05.167938+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:58:52.605804+0000","last_clean_scrub_stamp":"2024-09-16T12:58:52.605804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"173'729","reported_seq":7378,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:18.692579+0000","last_change":"2024-09-16T13:02:18.692548+0000","last_active":"2024-09-16T13:02:18.692579+0000","last_peered":"2024-09-16T13:02:18.692579+0000","last_clean":"2024-09-16T13:02:18.692579+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:02:18.692579+0000","last_undegraded":"2024-09-16T13:02:18.692579+0000","last_fullsized":"2024-09-16T13:02:18.692579+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'729","last_scrub_stamp":"2024-09-16T13:02:18.692462+0000","last_deep_scrub":"173'729","last_deep_scrub_stamp":"2024-09-16T13:02:18.692462+0000","last_clean_scrub_stamp":"2024-09-16T13:02:18.692462+0000","objects_scrubbed":39,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:01.085263+0000","scrub_duration":0.052170814000000003,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":172,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:24.116051+0000","last_change":"2024-09-16T13:02:24.115983+0000","last_active":"2024-09-16T13:02:24.116051+0000","last_peered":"2024-09-16T13:02:24.116051+0000","last_clean":"2024-09-16T13:02:24.116051+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:02:24.116051+0000","last_undegraded":"2024-09-16T13:02:24.116051+0000","last_fullsized":"2024-09-16T13:02:24.116051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:24.115743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:24.115743+0000","last_clean_scrub_stamp":"2024-09-16T13:02:24.115743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:30.842922+0000","scrub_duration":0.0041725199999999999,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"173'712","reported_seq":3256,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:16.279082+0000","last_change":"2024-09-16T13:02:16.279048+0000","last_active":"2024-09-16T13:02:16.279082+0000","last_peered":"2024-09-16T13:02:16.279082+0000","last_clean":"2024-09-16T13:02:16.279082+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:02:16.279082+0000","last_undegraded":"2024-09-16T13:02:16.279082+0000","last_fullsized":"2024-09-16T13:02:16.279082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'712","last_scrub_stamp":"2024-09-16T13:02:16.278933+0000","last_deep_scrub":"173'712","last_deep_scrub_stamp":"2024-09-16T13:02:16.278933+0000","last_clean_scrub_stamp":"2024-09-16T13:02:16.278933+0000","objects_scrubbed":25,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:30.813255+0000","scrub_duration":0.032644602000000002,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:20.717582+0000","last_change":"2024-09-16T13:02:20.717489+0000","last_active":"2024-09-16T13:02:20.717582+0000","last_peered":"2024-09-16T13:02:20.717582+0000","last_clean":"2024-09-16T13:02:20.717582+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:02:20.717582+0000","last_undegraded":"2024-09-16T13:02:20.717582+0000","last_fullsized":"2024-09-16T13:02:20.717582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:20.717253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:20.717253+0000","last_clean_scrub_stamp":"2024-09-16T13:02:20.717253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:11.441903+0000","scrub_duration":0.0035017,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"173'621","reported_seq":17157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:22.772824+0000","last_change":"2024-09-16T13:02:22.772771+0000","last_active":"2024-09-16T13:02:22.772824+0000","last_peered":"2024-09-16T13:02:22.772824+0000","last_clean":"2024-09-16T13:02:22.772824+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:02:22.772824+0000","last_undegraded":"2024-09-16T13:02:22.772824+0000","last_fullsized":"2024-09-16T13:02:22.772824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'621","last_scrub_stamp":"2024-09-16T13:02:22.772658+0000","last_deep_scrub":"173'621","last_deep_scrub_stamp":"2024-09-16T13:02:22.772658+0000","last_clean_scrub_stamp":"2024-09-16T13:02:22.772658+0000","objects_scrubbed":35,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:29:47.732119+0000","scrub_duration":0.041035980999999999,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":135,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:17.956789+0000","last_change":"2024-09-16T13:02:17.956728+0000","last_active":"2024-09-16T13:02:17.956789+0000","last_peered":"2024-09-16T13:02:17.956789+0000","last_clean":"2024-09-16T13:02:17.956789+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:02:17.956789+0000","last_undegraded":"2024-09-16T13:02:17.956789+0000","last_fullsized":"2024-09-16T13:02:17.956789+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:17.956476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:17.956476+0000","last_clean_scrub_stamp":"2024-09-16T13:02:17.956476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:37:47.411233+0000","scrub_duration":0.0037598750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"173'3","reported_seq":203,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:17.675641+0000","last_change":"2024-09-16T13:02:17.675529+0000","last_active":"2024-09-16T13:02:17.675641+0000","last_peered":"2024-09-16T13:02:17.675641+0000","last_clean":"2024-09-16T13:02:17.675641+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:02:17.675641+0000","last_undegraded":"2024-09-16T13:02:17.675641+0000","last_fullsized":"2024-09-16T13:02:17.675641+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'3","last_scrub_stamp":"2024-09-16T13:02:17.675286+0000","last_deep_scrub":"173'3","last_deep_scrub_stamp":"2024-09-16T13:02:17.675286+0000","last_clean_scrub_stamp":"2024-09-16T13:02:17.675286+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:13:08.123285+0000","scrub_duration":0.0068102479999999996,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"173'529","reported_seq":17977,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:19.662954+0000","last_change":"2024-09-16T13:02:19.662868+0000","last_active":"2024-09-16T13:02:19.662954+0000","last_peered":"2024-09-16T13:02:19.662954+0000","last_clean":"2024-09-16T13:02:19.662954+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:02:19.662954+0000","last_undegraded":"2024-09-16T13:02:19.662954+0000","last_fullsized":"2024-09-16T13:02:19.662954+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'529","last_scrub_stamp":"2024-09-16T13:02:19.662582+0000","last_deep_scrub":"173'529","last_deep_scrub_stamp":"2024-09-16T13:02:19.662582+0000","last_clean_scrub_stamp":"2024-09-16T13:02:19.662582+0000","objects_scrubbed":28,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:42:08.144768+0000","scrub_duration":0.033100588,"objects_trimmed":0,"snaptrim_duration":0.0013411569999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":133,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:15.720232+0000","last_change":"2024-09-16T13:02:15.720190+0000","last_active":"2024-09-16T13:02:15.720232+0000","last_peered":"2024-09-16T13:02:15.720232+0000","last_clean":"2024-09-16T13:02:15.720232+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:02:15.720232+0000","last_undegraded":"2024-09-16T13:02:15.720232+0000","last_fullsized":"2024-09-16T13:02:15.720232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:15.720010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:15.720010+0000","last_clean_scrub_stamp":"2024-09-16T13:02:15.720010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:31:53.846022+0000","scrub_duration":0.0021724029999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:13.436798+0000","last_change":"2024-09-16T13:02:13.436743+0000","last_active":"2024-09-16T13:02:13.436798+0000","last_peered":"2024-09-16T13:02:13.436798+0000","last_clean":"2024-09-16T13:02:13.436798+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:02:13.436798+0000","last_undegraded":"2024-09-16T13:02:13.436798+0000","last_fullsized":"2024-09-16T13:02:13.436798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:13.436562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:13.436562+0000","last_clean_scrub_stamp":"2024-09-16T13:02:13.436562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:39.754795+0000","scrub_duration":0.002781338,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"173'429","reported_seq":2808,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:18.681632+0000","last_change":"2024-09-16T13:02:18.681604+0000","last_active":"2024-09-16T13:02:18.681632+0000","last_peered":"2024-09-16T13:02:18.681632+0000","last_clean":"2024-09-16T13:02:18.681632+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:02:18.681632+0000","last_undegraded":"2024-09-16T13:02:18.681632+0000","last_fullsized":"2024-09-16T13:02:18.681632+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'429","last_scrub_stamp":"2024-09-16T13:02:18.681524+0000","last_deep_scrub":"173'429","last_deep_scrub_stamp":"2024-09-16T13:02:18.681524+0000","last_clean_scrub_stamp":"2024-09-16T13:02:18.681524+0000","objects_scrubbed":22,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:47.140889+0000","scrub_duration":0.025023360000000001,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"173'2","reported_seq":181,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:13.237728+0000","last_change":"2024-09-16T13:02:13.237670+0000","last_active":"2024-09-16T13:02:13.237728+0000","last_peered":"2024-09-16T13:02:13.237728+0000","last_clean":"2024-09-16T13:02:13.237728+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:02:13.237728+0000","last_undegraded":"2024-09-16T13:02:13.237728+0000","last_fullsized":"2024-09-16T13:02:13.237728+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'2","last_scrub_stamp":"2024-09-16T13:02:13.237504+0000","last_deep_scrub":"173'2","last_deep_scrub_stamp":"2024-09-16T13:02:13.237504+0000","last_clean_scrub_stamp":"2024-09-16T13:02:13.237504+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:57.398898+0000","scrub_duration":0.0054730079999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":190,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:22.062442+0000","last_change":"2024-09-16T13:02:22.062291+0000","last_active":"2024-09-16T13:02:22.062442+0000","last_peered":"2024-09-16T13:02:22.062442+0000","last_clean":"2024-09-16T13:02:22.062442+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:02:22.062442+0000","last_undegraded":"2024-09-16T13:02:22.062442+0000","last_fullsized":"2024-09-16T13:02:22.062442+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:22.062042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:22.062042+0000","last_clean_scrub_stamp":"2024-09-16T13:02:22.062042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:00:49.859183+0000","scrub_duration":0.0037660020000000001,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"173'461","reported_seq":7361,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:24.239514+0000","last_change":"2024-09-16T13:02:24.239446+0000","last_active":"2024-09-16T13:02:24.239514+0000","last_peered":"2024-09-16T13:02:24.239514+0000","last_clean":"2024-09-16T13:02:24.239514+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:02:24.239514+0000","last_undegraded":"2024-09-16T13:02:24.239514+0000","last_fullsized":"2024-09-16T13:02:24.239514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'461","last_scrub_stamp":"2024-09-16T13:02:24.239312+0000","last_deep_scrub":"173'461","last_deep_scrub_stamp":"2024-09-16T13:02:24.239312+0000","last_clean_scrub_stamp":"2024-09-16T13:02:24.239312+0000","objects_scrubbed":27,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:13.225155+0000","scrub_duration":0.033396594000000002,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"173'8","reported_seq":173,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:15.629638+0000","last_change":"2024-09-16T13:02:15.629566+0000","last_active":"2024-09-16T13:02:15.629638+0000","last_peered":"2024-09-16T13:02:15.629638+0000","last_clean":"2024-09-16T13:02:15.629638+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:02:15.629638+0000","last_undegraded":"2024-09-16T13:02:15.629638+0000","last_fullsized":"2024-09-16T13:02:15.629638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'8","last_scrub_stamp":"2024-09-16T13:02:15.629406+0000","last_deep_scrub":"173'8","last_deep_scrub_stamp":"2024-09-16T13:02:15.629406+0000","last_clean_scrub_stamp":"2024-09-16T13:02:15.629406+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:29.057820+0000","scrub_duration":0.0082631130000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"173'2","reported_seq":176,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:19.669940+0000","last_change":"2024-09-16T13:02:19.669890+0000","last_active":"2024-09-16T13:02:19.669940+0000","last_peered":"2024-09-16T13:02:19.669940+0000","last_clean":"2024-09-16T13:02:19.669940+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:02:19.669940+0000","last_undegraded":"2024-09-16T13:02:19.669940+0000","last_fullsized":"2024-09-16T13:02:19.669940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'2","last_scrub_stamp":"2024-09-16T13:02:19.669782+0000","last_deep_scrub":"173'2","last_deep_scrub_stamp":"2024-09-16T13:02:19.669782+0000","last_clean_scrub_stamp":"2024-09-16T13:02:19.669782+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:49.379007+0000","scrub_duration":0.003447075,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"173'74","reported_seq":262,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:18.019075+0000","last_change":"2024-09-16T13:02:18.018975+0000","last_active":"2024-09-16T13:02:18.019075+0000","last_peered":"2024-09-16T13:02:18.019075+0000","last_clean":"2024-09-16T13:02:18.019075+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:02:18.019075+0000","last_undegraded":"2024-09-16T13:02:18.019075+0000","last_fullsized":"2024-09-16T13:02:18.019075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'74","last_scrub_stamp":"2024-09-16T13:02:18.018757+0000","last_deep_scrub":"173'74","last_deep_scrub_stamp":"2024-09-16T13:02:18.018757+0000","last_clean_scrub_stamp":"2024-09-16T13:02:18.018757+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:28.960500+0000","scrub_duration":0.0086403190000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"173'480","reported_seq":3768,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:16.703771+0000","last_change":"2024-09-16T13:02:16.703731+0000","last_active":"2024-09-16T13:02:16.703771+0000","last_peered":"2024-09-16T13:02:16.703771+0000","last_clean":"2024-09-16T13:02:16.703771+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:02:16.703771+0000","last_undegraded":"2024-09-16T13:02:16.703771+0000","last_fullsized":"2024-09-16T13:02:16.703771+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'480","last_scrub_stamp":"2024-09-16T13:02:16.703624+0000","last_deep_scrub":"173'480","last_deep_scrub_stamp":"2024-09-16T13:02:16.703624+0000","last_clean_scrub_stamp":"2024-09-16T13:02:16.703624+0000","objects_scrubbed":36,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:55:40.982386+0000","scrub_duration":0.038056722000000001,"objects_trimmed":0,"snaptrim_duration":0.000602565,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"173'1004","reported_seq":18821,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:24.643546+0000","last_change":"2024-09-16T13:02:24.643495+0000","last_active":"2024-09-16T13:02:24.643546+0000","last_peered":"2024-09-16T13:02:24.643546+0000","last_clean":"2024-09-16T13:02:24.643546+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:02:24.643546+0000","last_undegraded":"2024-09-16T13:02:24.643546+0000","last_fullsized":"2024-09-16T13:02:24.643546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'1004","last_scrub_stamp":"2024-09-16T13:02:24.643353+0000","last_deep_scrub":"173'1004","last_deep_scrub_stamp":"2024-09-16T13:02:24.643353+0000","last_clean_scrub_stamp":"2024-09-16T13:02:24.643353+0000","objects_scrubbed":27,"log_size":1004,"log_dups_size":0,"ondisk_log_size":1004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:53.688699+0000","scrub_duration":0.036443428,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"173'350","reported_seq":880,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:15.976074+0000","last_change":"2024-09-16T13:02:15.976035+0000","last_active":"2024-09-16T13:02:15.976074+0000","last_peered":"2024-09-16T13:02:15.976074+0000","last_clean":"2024-09-16T13:02:15.976074+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:02:15.976074+0000","last_undegraded":"2024-09-16T13:02:15.976074+0000","last_fullsized":"2024-09-16T13:02:15.976074+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'350","last_scrub_stamp":"2024-09-16T13:02:15.975922+0000","last_deep_scrub":"173'350","last_deep_scrub_stamp":"2024-09-16T13:02:15.975922+0000","last_clean_scrub_stamp":"2024-09-16T13:02:15.975922+0000","objects_scrubbed":22,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:04.216764+0000","scrub_duration":0.02335044,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"173'1406","reported_seq":7010,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:15.693430+0000","last_change":"2024-09-16T13:02:15.693401+0000","last_active":"2024-09-16T13:02:15.693430+0000","last_peered":"2024-09-16T13:02:15.693430+0000","last_clean":"2024-09-16T13:02:15.693430+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:02:15.693430+0000","last_undegraded":"2024-09-16T13:02:15.693430+0000","last_fullsized":"2024-09-16T13:02:15.693430+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'1406","last_scrub_stamp":"2024-09-16T13:02:15.693311+0000","last_deep_scrub":"173'1406","last_deep_scrub_stamp":"2024-09-16T13:02:15.693311+0000","last_clean_scrub_stamp":"2024-09-16T13:02:15.693311+0000","objects_scrubbed":26,"log_size":1406,"log_dups_size":0,"ondisk_log_size":1406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:53.307086+0000","scrub_duration":0.038083835000000003,"objects_trimmed":0,"snaptrim_duration":0.00093887699999999996,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"173'1008","reported_seq":6683,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:27.215381+0000","last_change":"2024-09-16T13:02:27.215326+0000","last_active":"2024-09-16T13:02:27.215381+0000","last_peered":"2024-09-16T13:02:27.215381+0000","last_clean":"2024-09-16T13:02:27.215381+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:02:27.215381+0000","last_undegraded":"2024-09-16T13:02:27.215381+0000","last_fullsized":"2024-09-16T13:02:27.215381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'1008","last_scrub_stamp":"2024-09-16T13:02:27.215175+0000","last_deep_scrub":"173'1008","last_deep_scrub_stamp":"2024-09-16T13:02:27.215175+0000","last_clean_scrub_stamp":"2024-09-16T13:02:27.215175+0000","objects_scrubbed":34,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:30.096453+0000","scrub_duration":0.044163098999999997,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"173'586","reported_seq":5395,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:26.983000+0000","last_change":"2024-09-16T13:02:26.982942+0000","last_active":"2024-09-16T13:02:26.983000+0000","last_peered":"2024-09-16T13:02:26.983000+0000","last_clean":"2024-09-16T13:02:26.983000+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:02:26.983000+0000","last_undegraded":"2024-09-16T13:02:26.983000+0000","last_fullsized":"2024-09-16T13:02:26.983000+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'586","last_scrub_stamp":"2024-09-16T13:02:26.982715+0000","last_deep_scrub":"173'586","last_deep_scrub_stamp":"2024-09-16T13:02:26.982715+0000","last_clean_scrub_stamp":"2024-09-16T13:02:26.982715+0000","objects_scrubbed":33,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:48.212992+0000","scrub_duration":0.039761856999999998,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"150'434","reported_seq":4406,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:05.167611+0000","last_change":"2024-09-16T13:01:38.027791+0000","last_active":"2024-09-16T13:02:05.167611+0000","last_peered":"2024-09-16T13:02:05.167611+0000","last_clean":"2024-09-16T13:02:05.167611+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:02:05.167611+0000","last_undegraded":"2024-09-16T13:02:05.167611+0000","last_fullsized":"2024-09-16T13:02:05.167611+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"150'876","reported_seq":4052,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:05.167833+0000","last_change":"2024-09-16T13:01:38.025990+0000","last_active":"2024-09-16T13:02:05.167833+0000","last_peered":"2024-09-16T13:02:05.167833+0000","last_clean":"2024-09-16T13:02:05.167833+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:02:05.167833+0000","last_undegraded":"2024-09-16T13:02:05.167833+0000","last_fullsized":"2024-09-16T13:02:05.167833+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"149'919","reported_seq":36750,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:05.168115+0000","last_change":"2024-09-16T13:01:38.113695+0000","last_active":"2024-09-16T13:02:05.168115+0000","last_peered":"2024-09-16T13:02:05.168115+0000","last_clean":"2024-09-16T13:02:05.168115+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:02:05.168115+0000","last_undegraded":"2024-09-16T13:02:05.168115+0000","last_fullsized":"2024-09-16T13:02:05.168115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T12:59:10.732611+0000","last_clean_scrub_stamp":"2024-09-16T12:59:10.732611+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21015,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":19850,"ondisk_log_size":19850,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192526,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301748,"kb_used_data":2028,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070092,"statfs":{"total":96636764160,"available":96327774208,"internally_reserved":0,"allocated":2076672,"data_stored":1051715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":5,"up_from":15,"seq":64424509490,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303552,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068288,"statfs":{"total":96636764160,"available":96325926912,"internally_reserved":0,"allocated":2088960,"data_stored":1051902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.85599999999999998},"min":{"1min":0.48099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1499999999999999,"5min":1.272,"15min":1.272},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.83999999999999997},"min":{"1min":0.55100000000000005,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0900000000000001,"5min":1.391,"15min":1.391},"last":1.071}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.97299999999999998,"15min":0.89200000000000002},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.333,"5min":14.275,"15min":14.275},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.58699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3600000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.94199999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.58099999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2430000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.996,"5min":1.071,"15min":1.103},"min":{"1min":0.61799999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4830000000000001,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":1.1100000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.071,"15min":1.0669999999999999},"min":{"1min":0.61899999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.2989999999999999,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.115},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6299999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.198}]},{"osd":4,"last update":"Mon Sep 16 13:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.1299999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.137,"15min":1.167},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3540000000000001,"5min":1.673,"15min":1.673},"last":1.159},{"interface":"front","average":{"1min":0.998,"5min":1.093,"15min":1.109},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.391,"5min":1.611,"15min":1.611},"last":1.175}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.077,"15min":1.137},"min":{"1min":0.65300000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4379999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.026},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1850000000000001,"15min":1.2450000000000001},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.673,"5min":1.748,"15min":1.748},"last":1.252}]}]},{"osd":4,"up_from":108,"seq":463856467989,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309432,"kb_used_data":2032,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062408,"statfs":{"total":96636764160,"available":96319905792,"internally_reserved":0,"allocated":2080768,"data_stored":1051734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.1060000000000001}]},{"osd":1,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.0429999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.149},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.073}]},{"osd":3,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.89000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":7,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.22}]}]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299392,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072448,"statfs":{"total":96636764160,"available":96330186752,"internally_reserved":0,"allocated":2088960,"data_stored":1051902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.995,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.052},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71799999999999997,"15min":0.66700000000000004},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0940000000000001,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.85599999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.232,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.51500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.103,"5min":1.528,"15min":1.528},"last":0.70399999999999996}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98399999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.49299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5740000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.90400000000000003}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.2869999999999999}]},{"osd":5,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.109,"15min":1.129},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8520000000000001,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.1060000000000001,"15min":1.1180000000000001},"min":{"1min":0.56000000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.1910000000000001}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1419999999999999,"15min":1.129},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.1830000000000001,"15min":1.1859999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.2110000000000003,"5min":4.2110000000000003,"15min":4.2110000000000003},"last":1.0840000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1299999999999999},"min":{"1min":0.61099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.77700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9830000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.3109999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299376,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072464,"statfs":{"total":96636764160,"available":96330203136,"internally_reserved":0,"allocated":2072576,"data_stored":1051222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.76400000000000001,"15min":0.747},"min":{"1min":0.314,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.0980000000000001,"5min":10.654,"15min":10.654},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.81999999999999995,"15min":0.77100000000000002},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.97899999999999998,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.85599999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.52200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99099999999999999,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.90900000000000003}]},{"osd":3,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.89200000000000002,"15min":0.80400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.2430000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.077,"5min":1.647,"15min":1.647},"last":1.04}]},{"osd":4,"last update":"Mon Sep 16 13:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.089}]},{"osd":5,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.093,"15min":1.1930000000000001},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.226,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.008},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0740000000000001,"15min":1.1439999999999999},"min":{"1min":0.60799999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.383,"5min":2.125,"15min":2.125},"last":1.1539999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.1080000000000001,"15min":1.165},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.466,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.2210000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1970000000000001,"15min":1.2470000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6779999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.254}]},{"osd":7,"last update":"Mon Sep 16 13:01:45 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.139,"15min":1.2290000000000001},"min":{"1min":0.63400000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.478,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.1539999999999999,"15min":1.2450000000000001},"min":{"1min":0.63700000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.538,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.1399999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303596,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068244,"statfs":{"total":96636764160,"available":96325881856,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.88600000000000001,"15min":0.84199999999999997},"min":{"1min":0.39100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.1899999999999999,"5min":9.407,"15min":9.407},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.92500000000000004}]},{"osd":2,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.85999999999999999,"15min":0.94599999999999995},"min":{"1min":0.46899999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.319,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0469999999999999,"5min":2.254,"15min":2.254},"last":0.73899999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.36399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83199999999999996,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":1.028},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4750000000000001,"5min":2.044,"15min":2.044},"last":0.86399999999999999}]},{"osd":4,"last update":"Mon Sep 16 13:01:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.038},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.0169999999999999}]},{"osd":5,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0149999999999999,"15min":1.0640000000000001},"min":{"1min":0.56399999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4730000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":0.999,"5min":1.133,"15min":1.2050000000000001},"min":{"1min":0.60999999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.325,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.163}]},{"osd":6,"last update":"Mon Sep 16 13:00:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.1890000000000001,"15min":1.236},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.2809999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:01:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.129,"15min":1.177},"min":{"1min":0.745,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.327,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.228},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1599999999999999},"min":{"1min":0.70099999999999996,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.427,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.2509999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542195,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303596,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068244,"statfs":{"total":96636764160,"available":96325881856,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5920000000000001,"5min":1.73,"15min":1.73},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.145,"15min":1.26},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3080000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":0.92300000000000004}]},{"osd":1,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0249999999999999,"15min":1.1559999999999999},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.208,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.56499999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5149999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.0249999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0569999999999999},"min":{"1min":0.55400000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.298,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.143,"15min":1.2030000000000001},"min":{"1min":0.50900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1839999999999999,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.1220000000000001}]},{"osd":3,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0109999999999999,"15min":1.0720000000000001},"min":{"1min":0.47299999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.984,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.873},{"interface":"front","average":{"1min":1.028,"5min":1.2010000000000001,"15min":1.298},"min":{"1min":0.66900000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9299999999999999,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.2609999999999999}]},{"osd":4,"last update":"Mon Sep 16 13:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.22}]},{"osd":5,"last update":"Mon Sep 16 13:01:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.1759999999999999,"15min":1.3029999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.135,"5min":6.282,"15min":6.282},"last":1.157},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.93600000000000005,"15min":0.97799999999999998},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.2549999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.1799999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:00:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.4930000000000001,"15min":1.573},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.7829999999999999,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.069,"15min":1.097},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.3640000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303544,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068296,"statfs":{"total":96636764160,"available":96325935104,"internally_reserved":0,"allocated":2080768,"data_stored":1051734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.73999999999999999},"min":{"1min":0.38100000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.97799999999999998,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.628},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.254,"15min":0.254},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.69099999999999995}]},{"osd":2,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72799999999999998,"15min":0.77600000000000002},"min":{"1min":0.35699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.90400000000000003,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.181,"5min":3.024,"15min":3.024},"last":0.82599999999999996}]},{"osd":3,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.95999999999999996,"15min":1.034},"min":{"1min":0.53600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2250000000000001,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":1.077},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.49199999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.151,"5min":3.544,"15min":3.544},"last":0.872}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.294}]},{"osd":5,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.25,"15min":1.3979999999999999},"min":{"1min":0.55700000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.619,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0600000000000001,"15min":1.0840000000000001},"min":{"1min":0.78700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.423,"5min":2.5,"15min":2.5},"last":0.97299999999999998}]},{"osd":6,"last update":"Mon Sep 16 13:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.145,"15min":1.165},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":1.145}]},{"osd":7,"last update":"Mon Sep 16 13:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1279999999999999,"15min":1.1459999999999999},"min":{"1min":0.879,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.575,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.226},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.139,"15min":1.181},"min":{"1min":0.68700000000000006,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.534,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.264}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:02:36.258 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.258 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.258 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.258 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.259 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.259 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.259 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.259 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T12:58:52.605804+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=58, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.259 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.259 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.260 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-16T12:59:10.732611+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=12, tm_min=59, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=13, tm_min=2, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T13:02:36.260 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T13:02:56.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T13:02:56.574 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T13:02:56.577 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T13:02:56.589 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-16T13:02:56.306333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590740,"num_objects":1007,"num_object_clones":0,"num_object_copies":2014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":995,"num_read":336325,"num_read_kb":648370,"num_write":21122,"num_write_kb":309528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":32808300,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":334,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20282,"ondisk_log_size":20282,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424252,"kb_used_data":16380,"kb_used_omap":103,"kb_used_meta":2407704,"kb_avail":752550468,"statfs":{"total":773094113280,"available":770611679232,"internally_reserved":0,"allocated":16773120,"data_stored":9594707,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":105908,"internal_metadata":2465489484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004155"},"pg_stats":[{"pgid":"4.1f","version":"173'363","reported_seq":4264,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:21.702577+0000","last_change":"2024-09-16T13:02:21.702540+0000","last_active":"2024-09-16T13:02:21.702577+0000","last_peered":"2024-09-16T13:02:21.702577+0000","last_clean":"2024-09-16T13:02:21.702577+0000","last_became_active":"2024-09-16T13:01:34.979710+0000","last_became_peered":"2024-09-16T13:01:34.979710+0000","last_unstale":"2024-09-16T13:02:21.702577+0000","last_undegraded":"2024-09-16T13:02:21.702577+0000","last_fullsized":"2024-09-16T13:02:21.702577+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'363","last_scrub_stamp":"2024-09-16T13:02:21.702427+0000","last_deep_scrub":"173'363","last_deep_scrub_stamp":"2024-09-16T13:02:21.702427+0000","last_clean_scrub_stamp":"2024-09-16T13:02:21.702427+0000","objects_scrubbed":24,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:15.774917+0000","scrub_duration":0.024839829000000001,"objects_trimmed":0,"snaptrim_duration":0.0015622850000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":3949,"num_read_kb":19343,"num_write":333,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"173'551","reported_seq":17283,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:33.185811+0000","last_change":"2024-09-16T13:02:33.185762+0000","last_active":"2024-09-16T13:02:33.185811+0000","last_peered":"2024-09-16T13:02:33.185811+0000","last_clean":"2024-09-16T13:02:33.185811+0000","last_became_active":"2024-09-16T12:59:11.907638+0000","last_became_peered":"2024-09-16T12:59:11.907638+0000","last_unstale":"2024-09-16T13:02:33.185811+0000","last_undegraded":"2024-09-16T13:02:33.185811+0000","last_fullsized":"2024-09-16T13:02:33.185811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'551","last_scrub_stamp":"2024-09-16T13:02:33.185585+0000","last_deep_scrub":"173'551","last_deep_scrub_stamp":"2024-09-16T13:02:33.185585+0000","last_clean_scrub_stamp":"2024-09-16T13:02:33.185585+0000","objects_scrubbed":37,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:13.413605+0000","scrub_duration":0.046236735000000001,"objects_trimmed":0,"snaptrim_duration":0.001036941,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":17405,"num_read_kb":19365,"num_write":475,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"173'453","reported_seq":5015,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:19.621025+0000","last_change":"2024-09-16T13:02:19.620977+0000","last_active":"2024-09-16T13:02:19.621025+0000","last_peered":"2024-09-16T13:02:19.621025+0000","last_clean":"2024-09-16T13:02:19.621025+0000","last_became_active":"2024-09-16T12:59:40.999973+0000","last_became_peered":"2024-09-16T12:59:40.999973+0000","last_unstale":"2024-09-16T13:02:19.621025+0000","last_undegraded":"2024-09-16T13:02:19.621025+0000","last_fullsized":"2024-09-16T13:02:19.621025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'453","last_scrub_stamp":"2024-09-16T13:02:19.620828+0000","last_deep_scrub":"173'453","last_deep_scrub_stamp":"2024-09-16T13:02:19.620828+0000","last_clean_scrub_stamp":"2024-09-16T13:02:19.620828+0000","objects_scrubbed":20,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:00.490290+0000","scrub_duration":0.027059613999999999,"objects_trimmed":0,"snaptrim_duration":0.0021769559999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5128,"num_read_kb":15486,"num_write":348,"num_write_kb":11074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":11721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"173'1210","reported_seq":8064,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:16.511218+0000","last_change":"2024-09-16T13:02:16.511154+0000","last_active":"2024-09-16T13:02:16.511218+0000","last_peered":"2024-09-16T13:02:16.511218+0000","last_clean":"2024-09-16T13:02:16.511218+0000","last_became_active":"2024-09-16T12:59:40.996168+0000","last_became_peered":"2024-09-16T12:59:40.996168+0000","last_unstale":"2024-09-16T13:02:16.511218+0000","last_undegraded":"2024-09-16T13:02:16.511218+0000","last_fullsized":"2024-09-16T13:02:16.511218+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'1210","last_scrub_stamp":"2024-09-16T13:02:16.510938+0000","last_deep_scrub":"173'1210","last_deep_scrub_stamp":"2024-09-16T13:02:16.510938+0000","last_clean_scrub_stamp":"2024-09-16T13:02:16.510938+0000","objects_scrubbed":39,"log_size":1210,"log_dups_size":0,"ondisk_log_size":1210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:09.257774+0000","scrub_duration":0.059094725000000001,"objects_trimmed":0,"snaptrim_duration":0.000799922,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":10305,"num_read_kb":15884,"num_write":1540,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":63932,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"173'525","reported_seq":22083,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:36.076260+0000","last_change":"2024-09-16T13:02:36.076226+0000","last_active":"2024-09-16T13:02:36.076260+0000","last_peered":"2024-09-16T13:02:36.076260+0000","last_clean":"2024-09-16T13:02:36.076260+0000","last_became_active":"2024-09-16T13:00:59.710070+0000","last_became_peered":"2024-09-16T13:00:59.710070+0000","last_unstale":"2024-09-16T13:02:36.076260+0000","last_undegraded":"2024-09-16T13:02:36.076260+0000","last_fullsized":"2024-09-16T13:02:36.076260+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'525","last_scrub_stamp":"2024-09-16T13:02:36.076094+0000","last_deep_scrub":"173'525","last_deep_scrub_stamp":"2024-09-16T13:02:36.076094+0000","last_clean_scrub_stamp":"2024-09-16T13:02:36.076094+0000","objects_scrubbed":32,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:14.550880+0000","scrub_duration":0.038182726,"objects_trimmed":0,"snaptrim_duration":0.002491568,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":22918,"num_read_kb":45082,"num_write":436,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":823478,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"173'716","reported_seq":7185,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:37.095563+0000","last_change":"2024-09-16T13:02:37.095520+0000","last_active":"2024-09-16T13:02:37.095563+0000","last_peered":"2024-09-16T13:02:37.095563+0000","last_clean":"2024-09-16T13:02:37.095563+0000","last_became_active":"2024-09-16T13:01:00.707899+0000","last_became_peered":"2024-09-16T13:01:00.707899+0000","last_unstale":"2024-09-16T13:02:37.095563+0000","last_undegraded":"2024-09-16T13:02:37.095563+0000","last_fullsized":"2024-09-16T13:02:37.095563+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'716","last_scrub_stamp":"2024-09-16T13:02:37.095391+0000","last_deep_scrub":"173'716","last_deep_scrub_stamp":"2024-09-16T13:02:37.095391+0000","last_clean_scrub_stamp":"2024-09-16T13:02:37.095391+0000","objects_scrubbed":31,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:54.067617+0000","scrub_duration":0.038643886000000002,"objects_trimmed":0,"snaptrim_duration":0.0022896660000000001,"stat_sum":{"num_bytes":19,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":7910,"num_read_kb":15955,"num_write":889,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":2364174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"173'427","reported_seq":5514,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:24.649076+0000","last_change":"2024-09-16T13:02:24.649040+0000","last_active":"2024-09-16T13:02:24.649076+0000","last_peered":"2024-09-16T13:02:24.649076+0000","last_clean":"2024-09-16T13:02:24.649076+0000","last_became_active":"2024-09-16T12:59:11.755674+0000","last_became_peered":"2024-09-16T12:59:11.755674+0000","last_unstale":"2024-09-16T13:02:24.649076+0000","last_undegraded":"2024-09-16T13:02:24.649076+0000","last_fullsized":"2024-09-16T13:02:24.649076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'427","last_scrub_stamp":"2024-09-16T13:02:24.648909+0000","last_deep_scrub":"173'427","last_deep_scrub_stamp":"2024-09-16T13:02:24.648909+0000","last_clean_scrub_stamp":"2024-09-16T13:02:24.648909+0000","objects_scrubbed":28,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:23.167553+0000","scrub_duration":0.034920038,"objects_trimmed":0,"snaptrim_duration":0.0019791140000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4896,"num_read_kb":18655,"num_write":338,"num_write_kb":13665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"173'468","reported_seq":6496,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:18.944855+0000","last_change":"2024-09-16T13:02:18.944778+0000","last_active":"2024-09-16T13:02:18.944855+0000","last_peered":"2024-09-16T13:02:18.944855+0000","last_clean":"2024-09-16T13:02:18.944855+0000","last_became_active":"2024-09-16T13:01:34.981897+0000","last_became_peered":"2024-09-16T13:01:34.981897+0000","last_unstale":"2024-09-16T13:02:18.944855+0000","last_undegraded":"2024-09-16T13:02:18.944855+0000","last_fullsized":"2024-09-16T13:02:18.944855+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'468","last_scrub_stamp":"2024-09-16T13:02:18.944663+0000","last_deep_scrub":"173'468","last_deep_scrub_stamp":"2024-09-16T13:02:18.944663+0000","last_clean_scrub_stamp":"2024-09-16T13:02:18.944663+0000","objects_scrubbed":28,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:07.109499+0000","scrub_duration":0.035887090000000003,"objects_trimmed":0,"snaptrim_duration":0.001028993,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":6364,"num_read_kb":20390,"num_write":480,"num_write_kb":10098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":5559671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"173'925","reported_seq":7679,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:21.689278+0000","last_change":"2024-09-16T13:02:21.689215+0000","last_active":"2024-09-16T13:02:21.689278+0000","last_peered":"2024-09-16T13:02:21.689278+0000","last_clean":"2024-09-16T13:02:21.689278+0000","last_became_active":"2024-09-16T12:59:11.908168+0000","last_became_peered":"2024-09-16T12:59:11.908168+0000","last_unstale":"2024-09-16T13:02:21.689278+0000","last_undegraded":"2024-09-16T13:02:21.689278+0000","last_fullsized":"2024-09-16T13:02:21.689278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'925","last_scrub_stamp":"2024-09-16T13:02:21.689005+0000","last_deep_scrub":"173'925","last_deep_scrub_stamp":"2024-09-16T13:02:21.689005+0000","last_clean_scrub_stamp":"2024-09-16T13:02:21.689005+0000","objects_scrubbed":36,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:21.331466+0000","scrub_duration":0.053280920000000002,"objects_trimmed":0,"snaptrim_duration":0.0028247699999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":10903,"num_read_kb":29188,"num_write":1053,"num_write_kb":16685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"173'546","reported_seq":3360,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:31.185001+0000","last_change":"2024-09-16T13:02:31.184950+0000","last_active":"2024-09-16T13:02:31.185001+0000","last_peered":"2024-09-16T13:02:31.185001+0000","last_clean":"2024-09-16T13:02:31.185001+0000","last_became_active":"2024-09-16T12:59:11.765325+0000","last_became_peered":"2024-09-16T12:59:11.765325+0000","last_unstale":"2024-09-16T13:02:31.185001+0000","last_undegraded":"2024-09-16T13:02:31.185001+0000","last_fullsized":"2024-09-16T13:02:31.185001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'546","last_scrub_stamp":"2024-09-16T13:02:31.184830+0000","last_deep_scrub":"173'546","last_deep_scrub_stamp":"2024-09-16T13:02:31.184830+0000","last_clean_scrub_stamp":"2024-09-16T13:02:31.184830+0000","objects_scrubbed":38,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:41:34.292941+0000","scrub_duration":0.041164247000000001,"objects_trimmed":0,"snaptrim_duration":0.000661749,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":5093,"num_read_kb":12372,"num_write":424,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"173'555","reported_seq":16258,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:23.633663+0000","last_change":"2024-09-16T13:02:23.633623+0000","last_active":"2024-09-16T13:02:23.633663+0000","last_peered":"2024-09-16T13:02:23.633663+0000","last_clean":"2024-09-16T13:02:23.633663+0000","last_became_active":"2024-09-16T12:59:11.909680+0000","last_became_peered":"2024-09-16T12:59:11.909680+0000","last_unstale":"2024-09-16T13:02:23.633663+0000","last_undegraded":"2024-09-16T13:02:23.633663+0000","last_fullsized":"2024-09-16T13:02:23.633663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'555","last_scrub_stamp":"2024-09-16T13:02:23.633505+0000","last_deep_scrub":"173'555","last_deep_scrub_stamp":"2024-09-16T13:02:23.633505+0000","last_clean_scrub_stamp":"2024-09-16T13:02:23.633505+0000","objects_scrubbed":32,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:25.486925+0000","scrub_duration":0.037584139000000003,"objects_trimmed":0,"snaptrim_duration":0.0011197990000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":16431,"num_read_kb":17685,"num_write":562,"num_write_kb":5623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"173'607","reported_seq":23267,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:22.622952+0000","last_change":"2024-09-16T13:02:22.622911+0000","last_active":"2024-09-16T13:02:22.622952+0000","last_peered":"2024-09-16T13:02:22.622952+0000","last_clean":"2024-09-16T13:02:22.622952+0000","last_became_active":"2024-09-16T12:59:40.999977+0000","last_became_peered":"2024-09-16T12:59:40.999977+0000","last_unstale":"2024-09-16T13:02:22.622952+0000","last_undegraded":"2024-09-16T13:02:22.622952+0000","last_fullsized":"2024-09-16T13:02:22.622952+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'607","last_scrub_stamp":"2024-09-16T13:02:22.622800+0000","last_deep_scrub":"173'607","last_deep_scrub_stamp":"2024-09-16T13:02:22.622800+0000","last_clean_scrub_stamp":"2024-09-16T13:02:22.622800+0000","objects_scrubbed":28,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:07:41.667158+0000","scrub_duration":0.035254847999999998,"objects_trimmed":0,"snaptrim_duration":0.0023420649999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":23541,"num_read_kb":35095,"num_write":590,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":583246,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"173'539","reported_seq":6271,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:41.063761+0000","last_change":"2024-09-16T13:02:41.063692+0000","last_active":"2024-09-16T13:02:41.063761+0000","last_peered":"2024-09-16T13:02:41.063761+0000","last_clean":"2024-09-16T13:02:41.063761+0000","last_became_active":"2024-09-16T13:01:34.985115+0000","last_became_peered":"2024-09-16T13:01:34.985115+0000","last_unstale":"2024-09-16T13:02:41.063761+0000","last_undegraded":"2024-09-16T13:02:41.063761+0000","last_fullsized":"2024-09-16T13:02:41.063761+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'539","last_scrub_stamp":"2024-09-16T13:02:41.063569+0000","last_deep_scrub":"173'539","last_deep_scrub_stamp":"2024-09-16T13:02:41.063569+0000","last_clean_scrub_stamp":"2024-09-16T13:02:41.063569+0000","objects_scrubbed":27,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:51.808273+0000","scrub_duration":0.029693152,"objects_trimmed":0,"snaptrim_duration":0.0028061969999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":6659,"num_read_kb":27743,"num_write":488,"num_write_kb":42982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":5324914,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"173'416","reported_seq":3180,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:30.618061+0000","last_change":"2024-09-16T13:02:30.618028+0000","last_active":"2024-09-16T13:02:30.618061+0000","last_peered":"2024-09-16T13:02:30.618061+0000","last_clean":"2024-09-16T13:02:30.618061+0000","last_became_active":"2024-09-16T13:00:48.456389+0000","last_became_peered":"2024-09-16T13:00:48.456389+0000","last_unstale":"2024-09-16T13:02:30.618061+0000","last_undegraded":"2024-09-16T13:02:30.618061+0000","last_fullsized":"2024-09-16T13:02:30.618061+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'416","last_scrub_stamp":"2024-09-16T13:02:30.617916+0000","last_deep_scrub":"173'416","last_deep_scrub_stamp":"2024-09-16T13:02:30.617916+0000","last_clean_scrub_stamp":"2024-09-16T13:02:30.617916+0000","objects_scrubbed":45,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:06.920773+0000","scrub_duration":0.052651344000000003,"objects_trimmed":0,"snaptrim_duration":0.000444592,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":2826,"num_read_kb":14309,"num_write":384,"num_write_kb":3313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":2039185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"173'399","reported_seq":18875,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:29.622369+0000","last_change":"2024-09-16T13:02:29.622322+0000","last_active":"2024-09-16T13:02:29.622369+0000","last_peered":"2024-09-16T13:02:29.622369+0000","last_clean":"2024-09-16T13:02:29.622369+0000","last_became_active":"2024-09-16T12:59:11.906490+0000","last_became_peered":"2024-09-16T12:59:11.906490+0000","last_unstale":"2024-09-16T13:02:29.622369+0000","last_undegraded":"2024-09-16T13:02:29.622369+0000","last_fullsized":"2024-09-16T13:02:29.622369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'399","last_scrub_stamp":"2024-09-16T13:02:29.622174+0000","last_deep_scrub":"173'399","last_deep_scrub_stamp":"2024-09-16T13:02:29.622174+0000","last_clean_scrub_stamp":"2024-09-16T13:02:29.622174+0000","objects_scrubbed":36,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:35.334013+0000","scrub_duration":0.046116155999999998,"objects_trimmed":0,"snaptrim_duration":0.0020314790000000001,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":18569,"num_read_kb":29306,"num_write":402,"num_write_kb":9729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"173'376","reported_seq":5511,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:33.669832+0000","last_change":"2024-09-16T13:02:33.669799+0000","last_active":"2024-09-16T13:02:33.669832+0000","last_peered":"2024-09-16T13:02:33.669832+0000","last_clean":"2024-09-16T13:02:33.669832+0000","last_became_active":"2024-09-16T12:59:40.997582+0000","last_became_peered":"2024-09-16T12:59:40.997582+0000","last_unstale":"2024-09-16T13:02:33.669832+0000","last_undegraded":"2024-09-16T13:02:33.669832+0000","last_fullsized":"2024-09-16T13:02:33.669832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'376","last_scrub_stamp":"2024-09-16T13:02:33.669662+0000","last_deep_scrub":"173'376","last_deep_scrub_stamp":"2024-09-16T13:02:33.669662+0000","last_clean_scrub_stamp":"2024-09-16T13:02:33.669662+0000","objects_scrubbed":41,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:02.621542+0000","scrub_duration":0.043299137000000001,"objects_trimmed":0,"snaptrim_duration":0.0011833690000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5076,"num_read_kb":16250,"num_write":318,"num_write_kb":2928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:22.662212+0000","last_change":"2024-09-16T13:02:22.662137+0000","last_active":"2024-09-16T13:02:22.662212+0000","last_peered":"2024-09-16T13:02:22.662212+0000","last_clean":"2024-09-16T13:02:22.662212+0000","last_became_active":"2024-09-16T13:01:35.025168+0000","last_became_peered":"2024-09-16T13:01:35.025168+0000","last_unstale":"2024-09-16T13:02:22.662212+0000","last_undegraded":"2024-09-16T13:02:22.662212+0000","last_fullsized":"2024-09-16T13:02:22.662212+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:22.661894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:22.661894+0000","last_clean_scrub_stamp":"2024-09-16T13:02:22.661894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:38:51.676316+0000","scrub_duration":0.003126827,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"173'481","reported_seq":3817,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:28.977934+0000","last_change":"2024-09-16T13:02:28.977888+0000","last_active":"2024-09-16T13:02:28.977934+0000","last_peered":"2024-09-16T13:02:28.977934+0000","last_clean":"2024-09-16T13:02:28.977934+0000","last_became_active":"2024-09-16T13:00:58.690987+0000","last_became_peered":"2024-09-16T13:00:58.690987+0000","last_unstale":"2024-09-16T13:02:28.977934+0000","last_undegraded":"2024-09-16T13:02:28.977934+0000","last_fullsized":"2024-09-16T13:02:28.977934+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'481","last_scrub_stamp":"2024-09-16T13:02:28.977762+0000","last_deep_scrub":"173'481","last_deep_scrub_stamp":"2024-09-16T13:02:28.977762+0000","last_clean_scrub_stamp":"2024-09-16T13:02:28.977762+0000","objects_scrubbed":30,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:48:41.014327+0000","scrub_duration":0.031695378000000003,"objects_trimmed":0,"snaptrim_duration":0.0029558689999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":4502,"num_read_kb":3734,"num_write":367,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":70566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:31.746477+0000","last_change":"2024-09-16T13:02:31.746417+0000","last_active":"2024-09-16T13:02:31.746477+0000","last_peered":"2024-09-16T13:02:31.746477+0000","last_clean":"2024-09-16T13:02:31.746477+0000","last_became_active":"2024-09-16T13:01:34.980276+0000","last_became_peered":"2024-09-16T13:01:34.980276+0000","last_unstale":"2024-09-16T13:02:31.746477+0000","last_undegraded":"2024-09-16T13:02:31.746477+0000","last_fullsized":"2024-09-16T13:02:31.746477+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:31.746170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:31.746170+0000","last_clean_scrub_stamp":"2024-09-16T13:02:31.746170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:05.849694+0000","scrub_duration":0.0036730859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"173'729","reported_seq":7378,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:18.692579+0000","last_change":"2024-09-16T13:02:18.692548+0000","last_active":"2024-09-16T13:02:18.692579+0000","last_peered":"2024-09-16T13:02:18.692579+0000","last_clean":"2024-09-16T13:02:18.692579+0000","last_became_active":"2024-09-16T12:59:11.906612+0000","last_became_peered":"2024-09-16T12:59:11.906612+0000","last_unstale":"2024-09-16T13:02:18.692579+0000","last_undegraded":"2024-09-16T13:02:18.692579+0000","last_fullsized":"2024-09-16T13:02:18.692579+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'729","last_scrub_stamp":"2024-09-16T13:02:18.692462+0000","last_deep_scrub":"173'729","last_deep_scrub_stamp":"2024-09-16T13:02:18.692462+0000","last_clean_scrub_stamp":"2024-09-16T13:02:18.692462+0000","objects_scrubbed":39,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:01.085263+0000","scrub_duration":0.052170814000000003,"objects_trimmed":0,"snaptrim_duration":0.0021452559999999999,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":9231,"num_read_kb":16977,"num_write":598,"num_write_kb":6601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":172,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:24.116051+0000","last_change":"2024-09-16T13:02:24.115983+0000","last_active":"2024-09-16T13:02:24.116051+0000","last_peered":"2024-09-16T13:02:24.116051+0000","last_clean":"2024-09-16T13:02:24.116051+0000","last_became_active":"2024-09-16T12:58:53.633717+0000","last_became_peered":"2024-09-16T12:58:53.633717+0000","last_unstale":"2024-09-16T13:02:24.116051+0000","last_undegraded":"2024-09-16T13:02:24.116051+0000","last_fullsized":"2024-09-16T13:02:24.116051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:24.115743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:24.115743+0000","last_clean_scrub_stamp":"2024-09-16T13:02:24.115743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:30.842922+0000","scrub_duration":0.0041725199999999999,"objects_trimmed":0,"snaptrim_duration":0.00084633799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"173'712","reported_seq":3256,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:16.279082+0000","last_change":"2024-09-16T13:02:16.279048+0000","last_active":"2024-09-16T13:02:16.279082+0000","last_peered":"2024-09-16T13:02:16.279082+0000","last_clean":"2024-09-16T13:02:16.279082+0000","last_became_active":"2024-09-16T12:59:11.768686+0000","last_became_peered":"2024-09-16T12:59:11.768686+0000","last_unstale":"2024-09-16T13:02:16.279082+0000","last_undegraded":"2024-09-16T13:02:16.279082+0000","last_fullsized":"2024-09-16T13:02:16.279082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'712","last_scrub_stamp":"2024-09-16T13:02:16.278933+0000","last_deep_scrub":"173'712","last_deep_scrub_stamp":"2024-09-16T13:02:16.278933+0000","last_clean_scrub_stamp":"2024-09-16T13:02:16.278933+0000","objects_scrubbed":25,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:30.813255+0000","scrub_duration":0.032644602000000002,"objects_trimmed":0,"snaptrim_duration":0.00096409499999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":24,"num_read":3234,"num_read_kb":5788,"num_write":613,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:20.717582+0000","last_change":"2024-09-16T13:02:20.717489+0000","last_active":"2024-09-16T13:02:20.717582+0000","last_peered":"2024-09-16T13:02:20.717582+0000","last_clean":"2024-09-16T13:02:20.717582+0000","last_became_active":"2024-09-16T12:59:40.994051+0000","last_became_peered":"2024-09-16T12:59:40.994051+0000","last_unstale":"2024-09-16T13:02:20.717582+0000","last_undegraded":"2024-09-16T13:02:20.717582+0000","last_fullsized":"2024-09-16T13:02:20.717582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:20.717253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:20.717253+0000","last_clean_scrub_stamp":"2024-09-16T13:02:20.717253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:11.441903+0000","scrub_duration":0.0035017,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"173'621","reported_seq":17157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:22.772824+0000","last_change":"2024-09-16T13:02:22.772771+0000","last_active":"2024-09-16T13:02:22.772824+0000","last_peered":"2024-09-16T13:02:22.772824+0000","last_clean":"2024-09-16T13:02:22.772824+0000","last_became_active":"2024-09-16T12:59:40.996865+0000","last_became_peered":"2024-09-16T12:59:40.996865+0000","last_unstale":"2024-09-16T13:02:22.772824+0000","last_undegraded":"2024-09-16T13:02:22.772824+0000","last_fullsized":"2024-09-16T13:02:22.772824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'621","last_scrub_stamp":"2024-09-16T13:02:22.772658+0000","last_deep_scrub":"173'621","last_deep_scrub_stamp":"2024-09-16T13:02:22.772658+0000","last_clean_scrub_stamp":"2024-09-16T13:02:22.772658+0000","objects_scrubbed":35,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:29:47.732119+0000","scrub_duration":0.041035980999999999,"objects_trimmed":0,"snaptrim_duration":0.001570564,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":18157,"num_read_kb":20199,"num_write":666,"num_write_kb":8629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":491071,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":135,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:17.956789+0000","last_change":"2024-09-16T13:02:17.956728+0000","last_active":"2024-09-16T13:02:17.956789+0000","last_peered":"2024-09-16T13:02:17.956789+0000","last_clean":"2024-09-16T13:02:17.956789+0000","last_became_active":"2024-09-16T13:00:56.529175+0000","last_became_peered":"2024-09-16T13:00:56.529175+0000","last_unstale":"2024-09-16T13:02:17.956789+0000","last_undegraded":"2024-09-16T13:02:17.956789+0000","last_fullsized":"2024-09-16T13:02:17.956789+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:17.956476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:17.956476+0000","last_clean_scrub_stamp":"2024-09-16T13:02:17.956476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:37:47.411233+0000","scrub_duration":0.0037598750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"173'3","reported_seq":203,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:17.675641+0000","last_change":"2024-09-16T13:02:17.675529+0000","last_active":"2024-09-16T13:02:17.675641+0000","last_peered":"2024-09-16T13:02:17.675641+0000","last_clean":"2024-09-16T13:02:17.675641+0000","last_became_active":"2024-09-16T12:59:40.999498+0000","last_became_peered":"2024-09-16T12:59:40.999498+0000","last_unstale":"2024-09-16T13:02:17.675641+0000","last_undegraded":"2024-09-16T13:02:17.675641+0000","last_fullsized":"2024-09-16T13:02:17.675641+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'3","last_scrub_stamp":"2024-09-16T13:02:17.675286+0000","last_deep_scrub":"173'3","last_deep_scrub_stamp":"2024-09-16T13:02:17.675286+0000","last_clean_scrub_stamp":"2024-09-16T13:02:17.675286+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:13:08.123285+0000","scrub_duration":0.0068102479999999996,"objects_trimmed":0,"snaptrim_duration":0.002347324,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"173'529","reported_seq":17977,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:19.662954+0000","last_change":"2024-09-16T13:02:19.662868+0000","last_active":"2024-09-16T13:02:19.662954+0000","last_peered":"2024-09-16T13:02:19.662954+0000","last_clean":"2024-09-16T13:02:19.662954+0000","last_became_active":"2024-09-16T13:01:34.979177+0000","last_became_peered":"2024-09-16T13:01:34.979177+0000","last_unstale":"2024-09-16T13:02:19.662954+0000","last_undegraded":"2024-09-16T13:02:19.662954+0000","last_fullsized":"2024-09-16T13:02:19.662954+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'529","last_scrub_stamp":"2024-09-16T13:02:19.662582+0000","last_deep_scrub":"173'529","last_deep_scrub_stamp":"2024-09-16T13:02:19.662582+0000","last_clean_scrub_stamp":"2024-09-16T13:02:19.662582+0000","objects_scrubbed":28,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:42:08.144768+0000","scrub_duration":0.033100588,"objects_trimmed":0,"snaptrim_duration":0.0013411569999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":19440,"num_read_kb":23030,"num_write":500,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":54030,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":133,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:15.720232+0000","last_change":"2024-09-16T13:02:15.720190+0000","last_active":"2024-09-16T13:02:15.720232+0000","last_peered":"2024-09-16T13:02:15.720232+0000","last_clean":"2024-09-16T13:02:15.720232+0000","last_became_active":"2024-09-16T12:59:40.990983+0000","last_became_peered":"2024-09-16T12:59:40.990983+0000","last_unstale":"2024-09-16T13:02:15.720232+0000","last_undegraded":"2024-09-16T13:02:15.720232+0000","last_fullsized":"2024-09-16T13:02:15.720232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:15.720010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:15.720010+0000","last_clean_scrub_stamp":"2024-09-16T13:02:15.720010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:31:53.846022+0000","scrub_duration":0.0021724029999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:13.436798+0000","last_change":"2024-09-16T13:02:13.436743+0000","last_active":"2024-09-16T13:02:13.436798+0000","last_peered":"2024-09-16T13:02:13.436798+0000","last_clean":"2024-09-16T13:02:13.436798+0000","last_became_active":"2024-09-16T12:58:53.944066+0000","last_became_peered":"2024-09-16T12:58:53.944066+0000","last_unstale":"2024-09-16T13:02:13.436798+0000","last_undegraded":"2024-09-16T13:02:13.436798+0000","last_fullsized":"2024-09-16T13:02:13.436798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:13.436562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:13.436562+0000","last_clean_scrub_stamp":"2024-09-16T13:02:13.436562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:39.754795+0000","scrub_duration":0.002781338,"objects_trimmed":0,"snaptrim_duration":0.00082888199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"173'429","reported_seq":2808,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:18.681632+0000","last_change":"2024-09-16T13:02:18.681604+0000","last_active":"2024-09-16T13:02:18.681632+0000","last_peered":"2024-09-16T13:02:18.681632+0000","last_clean":"2024-09-16T13:02:18.681632+0000","last_became_active":"2024-09-16T12:59:40.995074+0000","last_became_peered":"2024-09-16T12:59:40.995074+0000","last_unstale":"2024-09-16T13:02:18.681632+0000","last_undegraded":"2024-09-16T13:02:18.681632+0000","last_fullsized":"2024-09-16T13:02:18.681632+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'429","last_scrub_stamp":"2024-09-16T13:02:18.681524+0000","last_deep_scrub":"173'429","last_deep_scrub_stamp":"2024-09-16T13:02:18.681524+0000","last_clean_scrub_stamp":"2024-09-16T13:02:18.681524+0000","objects_scrubbed":22,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:47.140889+0000","scrub_duration":0.025023360000000001,"objects_trimmed":0,"snaptrim_duration":0.00069419299999999996,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2825,"num_read_kb":19442,"num_write":423,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4233015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"173'3","reported_seq":182,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:13.237759+0000","last_change":"2024-09-16T13:02:13.237670+0000","last_active":"2024-09-16T13:02:13.237759+0000","last_peered":"2024-09-16T13:02:13.237759+0000","last_clean":"2024-09-16T13:02:13.237759+0000","last_became_active":"2024-09-16T13:00:56.528223+0000","last_became_peered":"2024-09-16T13:00:56.528223+0000","last_unstale":"2024-09-16T13:02:13.237759+0000","last_undegraded":"2024-09-16T13:02:13.237759+0000","last_fullsized":"2024-09-16T13:02:13.237759+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'2","last_scrub_stamp":"2024-09-16T13:02:13.237504+0000","last_deep_scrub":"173'2","last_deep_scrub_stamp":"2024-09-16T13:02:13.237504+0000","last_clean_scrub_stamp":"2024-09-16T13:02:13.237504+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:57.398898+0000","scrub_duration":0.0054730079999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":190,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:22.062442+0000","last_change":"2024-09-16T13:02:22.062291+0000","last_active":"2024-09-16T13:02:22.062442+0000","last_peered":"2024-09-16T13:02:22.062442+0000","last_clean":"2024-09-16T13:02:22.062442+0000","last_became_active":"2024-09-16T12:59:40.998044+0000","last_became_peered":"2024-09-16T12:59:40.998044+0000","last_unstale":"2024-09-16T13:02:22.062442+0000","last_undegraded":"2024-09-16T13:02:22.062442+0000","last_fullsized":"2024-09-16T13:02:22.062442+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T13:02:22.062042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T13:02:22.062042+0000","last_clean_scrub_stamp":"2024-09-16T13:02:22.062042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:00:49.859183+0000","scrub_duration":0.0037660020000000001,"objects_trimmed":0,"snaptrim_duration":0.00088090399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"173'461","reported_seq":7361,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:24.239514+0000","last_change":"2024-09-16T13:02:24.239446+0000","last_active":"2024-09-16T13:02:24.239514+0000","last_peered":"2024-09-16T13:02:24.239514+0000","last_clean":"2024-09-16T13:02:24.239514+0000","last_became_active":"2024-09-16T12:59:40.997430+0000","last_became_peered":"2024-09-16T12:59:40.997430+0000","last_unstale":"2024-09-16T13:02:24.239514+0000","last_undegraded":"2024-09-16T13:02:24.239514+0000","last_fullsized":"2024-09-16T13:02:24.239514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'461","last_scrub_stamp":"2024-09-16T13:02:24.239312+0000","last_deep_scrub":"173'461","last_deep_scrub_stamp":"2024-09-16T13:02:24.239312+0000","last_clean_scrub_stamp":"2024-09-16T13:02:24.239312+0000","objects_scrubbed":27,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:13.225155+0000","scrub_duration":0.033396594000000002,"objects_trimmed":0,"snaptrim_duration":0.00092258599999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7056,"num_read_kb":14221,"num_write":319,"num_write_kb":7395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":291278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"173'9","reported_seq":174,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:15.629692+0000","last_change":"2024-09-16T13:02:15.629566+0000","last_active":"2024-09-16T13:02:15.629692+0000","last_peered":"2024-09-16T13:02:15.629692+0000","last_clean":"2024-09-16T13:02:15.629692+0000","last_became_active":"2024-09-16T12:59:02.145684+0000","last_became_peered":"2024-09-16T12:59:02.145684+0000","last_unstale":"2024-09-16T13:02:15.629692+0000","last_undegraded":"2024-09-16T13:02:15.629692+0000","last_fullsized":"2024-09-16T13:02:15.629692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'8","last_scrub_stamp":"2024-09-16T13:02:15.629406+0000","last_deep_scrub":"173'8","last_deep_scrub_stamp":"2024-09-16T13:02:15.629406+0000","last_clean_scrub_stamp":"2024-09-16T13:02:15.629406+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:29.057820+0000","scrub_duration":0.0082631130000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":251,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"173'2","reported_seq":176,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:19.669940+0000","last_change":"2024-09-16T13:02:19.669890+0000","last_active":"2024-09-16T13:02:19.669940+0000","last_peered":"2024-09-16T13:02:19.669940+0000","last_clean":"2024-09-16T13:02:19.669940+0000","last_became_active":"2024-09-16T12:58:53.631548+0000","last_became_peered":"2024-09-16T12:58:53.631548+0000","last_unstale":"2024-09-16T13:02:19.669940+0000","last_undegraded":"2024-09-16T13:02:19.669940+0000","last_fullsized":"2024-09-16T13:02:19.669940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'2","last_scrub_stamp":"2024-09-16T13:02:19.669782+0000","last_deep_scrub":"173'2","last_deep_scrub_stamp":"2024-09-16T13:02:19.669782+0000","last_clean_scrub_stamp":"2024-09-16T13:02:19.669782+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:49.379007+0000","scrub_duration":0.003447075,"objects_trimmed":0,"snaptrim_duration":0.00051248400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"173'74","reported_seq":262,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:18.019075+0000","last_change":"2024-09-16T13:02:18.018975+0000","last_active":"2024-09-16T13:02:18.019075+0000","last_peered":"2024-09-16T13:02:18.019075+0000","last_clean":"2024-09-16T13:02:18.019075+0000","last_became_active":"2024-09-16T12:58:46.130820+0000","last_became_peered":"2024-09-16T12:58:46.130820+0000","last_unstale":"2024-09-16T13:02:18.019075+0000","last_undegraded":"2024-09-16T13:02:18.019075+0000","last_fullsized":"2024-09-16T13:02:18.019075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'74","last_scrub_stamp":"2024-09-16T13:02:18.018757+0000","last_deep_scrub":"173'74","last_deep_scrub_stamp":"2024-09-16T13:02:18.018757+0000","last_clean_scrub_stamp":"2024-09-16T13:02:18.018757+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:28.960500+0000","scrub_duration":0.0086403190000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"173'480","reported_seq":3768,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:16.703771+0000","last_change":"2024-09-16T13:02:16.703731+0000","last_active":"2024-09-16T13:02:16.703771+0000","last_peered":"2024-09-16T13:02:16.703771+0000","last_clean":"2024-09-16T13:02:16.703771+0000","last_became_active":"2024-09-16T13:01:34.981334+0000","last_became_peered":"2024-09-16T13:01:34.981334+0000","last_unstale":"2024-09-16T13:02:16.703771+0000","last_undegraded":"2024-09-16T13:02:16.703771+0000","last_fullsized":"2024-09-16T13:02:16.703771+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'480","last_scrub_stamp":"2024-09-16T13:02:16.703624+0000","last_deep_scrub":"173'480","last_deep_scrub_stamp":"2024-09-16T13:02:16.703624+0000","last_clean_scrub_stamp":"2024-09-16T13:02:16.703624+0000","objects_scrubbed":36,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:55:40.982386+0000","scrub_duration":0.038056722000000001,"objects_trimmed":0,"snaptrim_duration":0.000602565,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3583,"num_read_kb":12968,"num_write":378,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":4856395,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"173'1004","reported_seq":18821,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:24.643546+0000","last_change":"2024-09-16T13:02:24.643495+0000","last_active":"2024-09-16T13:02:24.643546+0000","last_peered":"2024-09-16T13:02:24.643546+0000","last_clean":"2024-09-16T13:02:24.643546+0000","last_became_active":"2024-09-16T12:59:11.909480+0000","last_became_peered":"2024-09-16T12:59:11.909480+0000","last_unstale":"2024-09-16T13:02:24.643546+0000","last_undegraded":"2024-09-16T13:02:24.643546+0000","last_fullsized":"2024-09-16T13:02:24.643546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'1004","last_scrub_stamp":"2024-09-16T13:02:24.643353+0000","last_deep_scrub":"173'1004","last_deep_scrub_stamp":"2024-09-16T13:02:24.643353+0000","last_clean_scrub_stamp":"2024-09-16T13:02:24.643353+0000","objects_scrubbed":27,"log_size":1004,"log_dups_size":0,"ondisk_log_size":1004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:53.688699+0000","scrub_duration":0.036443428,"objects_trimmed":0,"snaptrim_duration":0.00070440200000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":21300,"num_read_kb":27213,"num_write":1294,"num_write_kb":13866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"173'350","reported_seq":880,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:15.976074+0000","last_change":"2024-09-16T13:02:15.976035+0000","last_active":"2024-09-16T13:02:15.976074+0000","last_peered":"2024-09-16T13:02:15.976074+0000","last_clean":"2024-09-16T13:02:15.976074+0000","last_became_active":"2024-09-16T13:00:56.532226+0000","last_became_peered":"2024-09-16T13:00:56.532226+0000","last_unstale":"2024-09-16T13:02:15.976074+0000","last_undegraded":"2024-09-16T13:02:15.976074+0000","last_fullsized":"2024-09-16T13:02:15.976074+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'350","last_scrub_stamp":"2024-09-16T13:02:15.975922+0000","last_deep_scrub":"173'350","last_deep_scrub_stamp":"2024-09-16T13:02:15.975922+0000","last_clean_scrub_stamp":"2024-09-16T13:02:15.975922+0000","objects_scrubbed":22,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:04.216764+0000","scrub_duration":0.02335044,"objects_trimmed":0,"snaptrim_duration":0.00087673099999999995,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2947,"num_read_kb":6219,"num_write":307,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2633928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"173'1406","reported_seq":7010,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:15.693430+0000","last_change":"2024-09-16T13:02:15.693401+0000","last_active":"2024-09-16T13:02:15.693430+0000","last_peered":"2024-09-16T13:02:15.693430+0000","last_clean":"2024-09-16T13:02:15.693430+0000","last_became_active":"2024-09-16T13:01:34.981982+0000","last_became_peered":"2024-09-16T13:01:34.981982+0000","last_unstale":"2024-09-16T13:02:15.693430+0000","last_undegraded":"2024-09-16T13:02:15.693430+0000","last_fullsized":"2024-09-16T13:02:15.693430+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'1406","last_scrub_stamp":"2024-09-16T13:02:15.693311+0000","last_deep_scrub":"173'1406","last_deep_scrub_stamp":"2024-09-16T13:02:15.693311+0000","last_clean_scrub_stamp":"2024-09-16T13:02:15.693311+0000","objects_scrubbed":26,"log_size":1406,"log_dups_size":0,"ondisk_log_size":1406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:53.307086+0000","scrub_duration":0.038083835000000003,"objects_trimmed":0,"snaptrim_duration":0.00093887699999999996,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":11653,"num_read_kb":20789,"num_write":2045,"num_write_kb":5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":33178,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"173'1008","reported_seq":6683,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:27.215381+0000","last_change":"2024-09-16T13:02:27.215326+0000","last_active":"2024-09-16T13:02:27.215381+0000","last_peered":"2024-09-16T13:02:27.215381+0000","last_clean":"2024-09-16T13:02:27.215381+0000","last_became_active":"2024-09-16T12:59:40.997422+0000","last_became_peered":"2024-09-16T12:59:40.997422+0000","last_unstale":"2024-09-16T13:02:27.215381+0000","last_undegraded":"2024-09-16T13:02:27.215381+0000","last_fullsized":"2024-09-16T13:02:27.215381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'1008","last_scrub_stamp":"2024-09-16T13:02:27.215175+0000","last_deep_scrub":"173'1008","last_deep_scrub_stamp":"2024-09-16T13:02:27.215175+0000","last_clean_scrub_stamp":"2024-09-16T13:02:27.215175+0000","objects_scrubbed":34,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:30.096453+0000","scrub_duration":0.044163098999999997,"objects_trimmed":0,"snaptrim_duration":0.00055493399999999998,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":8784,"num_read_kb":20053,"num_write":1413,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16455,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"173'586","reported_seq":5395,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:26.983000+0000","last_change":"2024-09-16T13:02:26.982942+0000","last_active":"2024-09-16T13:02:26.983000+0000","last_peered":"2024-09-16T13:02:26.983000+0000","last_clean":"2024-09-16T13:02:26.983000+0000","last_became_active":"2024-09-16T13:00:58.694541+0000","last_became_peered":"2024-09-16T13:00:58.694541+0000","last_unstale":"2024-09-16T13:02:26.983000+0000","last_undegraded":"2024-09-16T13:02:26.983000+0000","last_fullsized":"2024-09-16T13:02:26.983000+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'586","last_scrub_stamp":"2024-09-16T13:02:26.982715+0000","last_deep_scrub":"173'586","last_deep_scrub_stamp":"2024-09-16T13:02:26.982715+0000","last_clean_scrub_stamp":"2024-09-16T13:02:26.982715+0000","objects_scrubbed":33,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:48.212992+0000","scrub_duration":0.039761856999999998,"objects_trimmed":0,"snaptrim_duration":0.001939903,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":5836,"num_read_kb":16184,"num_write":515,"num_write_kb":7010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1039042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"173'465","reported_seq":4448,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:33.028295+0000","last_change":"2024-09-16T13:02:33.028176+0000","last_active":"2024-09-16T13:02:33.028295+0000","last_peered":"2024-09-16T13:02:33.028295+0000","last_clean":"2024-09-16T13:02:33.028295+0000","last_became_active":"2024-09-16T13:00:58.692193+0000","last_became_peered":"2024-09-16T13:00:58.692193+0000","last_unstale":"2024-09-16T13:02:33.028295+0000","last_undegraded":"2024-09-16T13:02:33.028295+0000","last_fullsized":"2024-09-16T13:02:33.028295+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'465","last_scrub_stamp":"2024-09-16T13:02:33.028007+0000","last_deep_scrub":"173'465","last_deep_scrub_stamp":"2024-09-16T13:02:33.028007+0000","last_clean_scrub_stamp":"2024-09-16T13:02:33.028007+0000","objects_scrubbed":31,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:06.585034+0000","scrub_duration":0.040484397999999998,"objects_trimmed":0,"snaptrim_duration":0.002608244,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5156,"num_read_kb":15142,"num_write":416,"num_write_kb":2469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1136459,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"173'904","reported_seq":4091,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:40.079771+0000","last_change":"2024-09-16T13:02:40.079737+0000","last_active":"2024-09-16T13:02:40.079771+0000","last_peered":"2024-09-16T13:02:40.079771+0000","last_clean":"2024-09-16T13:02:40.079771+0000","last_became_active":"2024-09-16T13:01:34.985183+0000","last_became_peered":"2024-09-16T13:01:34.985183+0000","last_unstale":"2024-09-16T13:02:40.079771+0000","last_undegraded":"2024-09-16T13:02:40.079771+0000","last_fullsized":"2024-09-16T13:02:40.079771+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'904","last_scrub_stamp":"2024-09-16T13:02:40.079632+0000","last_deep_scrub":"173'904","last_deep_scrub_stamp":"2024-09-16T13:02:40.079632+0000","last_clean_scrub_stamp":"2024-09-16T13:02:40.079632+0000","objects_scrubbed":28,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:36:05.653837+0000","scrub_duration":0.037770314999999999,"objects_trimmed":0,"snaptrim_duration":0.002102039,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5413,"num_read_kb":7443,"num_write":1002,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":1103677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"173'950","reported_seq":36792,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T13:02:34.744848+0000","last_change":"2024-09-16T13:02:34.744805+0000","last_active":"2024-09-16T13:02:34.744848+0000","last_peered":"2024-09-16T13:02:34.744848+0000","last_clean":"2024-09-16T13:02:34.744848+0000","last_became_active":"2024-09-16T12:59:40.994173+0000","last_became_peered":"2024-09-16T12:59:40.994173+0000","last_unstale":"2024-09-16T13:02:34.744848+0000","last_undegraded":"2024-09-16T13:02:34.744848+0000","last_fullsized":"2024-09-16T13:02:34.744848+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'950","last_scrub_stamp":"2024-09-16T13:02:34.744675+0000","last_deep_scrub":"173'950","last_deep_scrub_stamp":"2024-09-16T13:02:34.744675+0000","last_clean_scrub_stamp":"2024-09-16T13:02:34.744675+0000","objects_scrubbed":31,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:42.598175+0000","scrub_duration":0.037599740999999999,"objects_trimmed":0,"snaptrim_duration":0.00077458699999999998,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":39169,"num_read_kb":66805,"num_write":1099,"num_write_kb":23502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":78847,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":992,"num_read":336259,"num_read_kb":648315,"num_write":21015,"num_write_kb":307894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1062,"num_bytes_recovered":32808262,"num_keys_recovered":160,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30457,"internal_metadata":0},"log_size":20191,"ondisk_log_size":20191,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":334,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":334,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18676,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":147,"seq":631360192530,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301748,"kb_used_data":2028,"kb_used_omap":57,"kb_used_meta":299654,"kb_avail":94070092,"statfs":{"total":96636764160,"available":96327774208,"internally_reserved":0,"allocated":2076672,"data_stored":1051715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":59390,"internal_metadata":306845698},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.095}]},{"osd":1,"last update":"Mon Sep 16 13:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.121}]},{"osd":2,"last update":"Mon Sep 16 13:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.98699999999999999}]},{"osd":3,"last update":"Mon Sep 16 13:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.2150000000000001}]},{"osd":4,"last update":"Mon Sep 16 13:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.1659999999999999}]},{"osd":5,"last update":"Mon Sep 16 13:02:34 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.3740000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:02:34 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.446},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.27}]}]},{"osd":5,"up_from":15,"seq":64424509495,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303556,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068284,"statfs":{"total":96636764160,"available":96325922816,"internally_reserved":0,"allocated":2093056,"data_stored":1051985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.83099999999999996,"15min":0.85899999999999999},"min":{"1min":0.52600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.23,"5min":1.272,"15min":1.272},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.83399999999999996,"15min":0.84599999999999997},"min":{"1min":0.57899999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2170000000000001,"5min":1.391,"15min":1.391},"last":1.2170000000000001}]},{"osd":1,"last update":"Mon Sep 16 13:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.89800000000000002},"min":{"1min":0.50900000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1719999999999999,"5min":14.275,"15min":14.275},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94099999999999995,"15min":0.90900000000000003},"min":{"1min":0.65300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3260000000000001,"5min":6.7370000000000001,"15min":6.7370000000000001},"last":0.97799999999999998}]},{"osd":2,"last update":"Mon Sep 16 13:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.91300000000000003,"15min":0.91600000000000004},"min":{"1min":0.68300000000000005,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.111,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.042,"15min":1.0940000000000001},"min":{"1min":0.71499999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2589999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Sep 16 13:02:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.044,"15min":1.0580000000000001},"min":{"1min":0.71599999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.3879999999999999,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.119,"5min":1.0840000000000001,"15min":1.1140000000000001},"min":{"1min":0.84799999999999998,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.272,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.01}]},{"osd":4,"last update":"Mon Sep 16 13:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0609999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.1339999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:02:53 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.05,"15min":1.1279999999999999},"min":{"1min":0.69299999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.29,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.1779999999999999,"15min":1.242},"min":{"1min":0.84999999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.468,"5min":1.748,"15min":1.748},"last":1.254}]}]},{"osd":4,"up_from":108,"seq":463856467994,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309436,"kb_used_data":2036,"kb_used_omap":35,"kb_used_meta":307356,"kb_avail":94062404,"statfs":{"total":96636764160,"available":96319901696,"internally_reserved":0,"allocated":2084864,"data_stored":1051817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36294,"internal_metadata":314733114},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.024},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.044}]},{"osd":1,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.1799999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.139}]},{"osd":3,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.90400000000000003}]},{"osd":5,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.64000000000000001}]},{"osd":6,"last update":"Mon Sep 16 13:02:35 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.96399999999999997}]},{"osd":7,"last update":"Mon Sep 16 13:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.123}]}]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299396,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072444,"statfs":{"total":96636764160,"available":96330182656,"internally_reserved":0,"allocated":2093056,"data_stored":1051985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.995,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71799999999999997,"15min":0.66700000000000004},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0940000000000001,"5min":6.2549999999999999,"15min":6.2549999999999999},"last":0.78100000000000003}]},{"osd":1,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.232,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.77700000000000002},"min":{"1min":0.51500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.103,"5min":1.528,"15min":1.528},"last":0.66400000000000003}]},{"osd":2,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98399999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.49299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5740000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.0309999999999999}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.161}]},{"osd":5,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.109,"15min":1.129},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8520000000000001,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1060000000000001,"15min":1.1180000000000001},"min":{"1min":0.56000000000000005,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.2390000000000001}]},{"osd":6,"last update":"Mon Sep 16 13:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.0489999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:01:53 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1299999999999999},"min":{"1min":0.61099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.321},{"interface":"front","average":{"1min":1.167,"5min":1.2050000000000001,"15min":1.1910000000000001},"min":{"1min":0.77700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9830000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.284}]}]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299376,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072464,"statfs":{"total":96636764160,"available":96330203136,"internally_reserved":0,"allocated":2072576,"data_stored":1051222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.746,"15min":0.74099999999999999},"min":{"1min":0.36399999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.92300000000000004,"5min":10.654,"15min":10.654},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83499999999999996,"15min":0.77600000000000002},"min":{"1min":0.46500000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.327,"5min":10.816000000000001,"15min":10.816000000000001},"last":0.73099999999999998}]},{"osd":1,"last update":"Mon Sep 16 13:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.42799999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2789999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.81599999999999995,"15min":0.83999999999999997},"min":{"1min":0.499,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97999999999999998,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.77000000000000002}]},{"osd":3,"last update":"Mon Sep 16 13:02:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.90200000000000002,"15min":0.80800000000000005},"min":{"1min":0.40300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.8420000000000001,"5min":18.117000000000001,"15min":18.117000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.85499999999999998,"15min":0.89500000000000002},"min":{"1min":0.55700000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5669999999999999,"5min":1.647,"15min":1.647},"last":0.86099999999999999}]},{"osd":4,"last update":"Mon Sep 16 13:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.212}]},{"osd":5,"last update":"Mon Sep 16 13:02:45 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.048,"15min":1.1779999999999999},"min":{"1min":0.629,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.516,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.056,"15min":1.1379999999999999},"min":{"1min":0.77300000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.772,"5min":2.125,"15min":2.125},"last":1.1020000000000001}]},{"osd":6,"last update":"Mon Sep 16 13:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.2969999999999999}]},{"osd":7,"last update":"Mon Sep 16 13:02:45 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.133,"15min":1.2270000000000001},"min":{"1min":0.69999999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.7350000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.357},{"interface":"front","average":{"1min":1.147,"5min":1.125,"15min":1.2350000000000001},"min":{"1min":0.65400000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.901,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.242}]}]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303600,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068240,"statfs":{"total":96636764160,"available":96325877760,"internally_reserved":0,"allocated":2138112,"data_stored":1642166,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 13:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.85799999999999998,"15min":0.83199999999999996},"min":{"1min":0.496,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.91200000000000003,"5min":9.407,"15min":9.407},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.40500000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92500000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.73999999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82899999999999996,"15min":0.93600000000000005},"min":{"1min":0.57899999999999996,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.125,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76000000000000001,"15min":0.80400000000000005},"min":{"1min":0.50900000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.071,"5min":2.254,"15min":2.254},"last":0.65400000000000003}]},{"osd":3,"last update":"Mon Sep 16 13:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.46700000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.93200000000000005,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.93899999999999995,"15min":1.0229999999999999},"min":{"1min":0.56299999999999994,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.57699999999999996}]},{"osd":4,"last update":"Mon Sep 16 13:01:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.97899999999999998}]},{"osd":5,"last update":"Mon Sep 16 13:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.046,"15min":1.0740000000000001},"min":{"1min":0.81799999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.873},{"interface":"front","average":{"1min":1.099,"5min":1.105,"15min":1.196},"min":{"1min":0.85599999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3919999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.0629999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:02:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.133},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.008}]},{"osd":7,"last update":"Mon Sep 16 13:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.119,"15min":1.173},"min":{"1min":0.91600000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4199999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1060000000000001,"15min":1.1579999999999999},"min":{"1min":0.90100000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.504,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":1.222}]}]},{"osd":7,"up_from":14,"seq":60129542199,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303596,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068244,"statfs":{"total":96636764160,"available":96325881856,"internally_reserved":0,"allocated":2134016,"data_stored":1642083,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.50600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5069999999999999,"5min":1.73,"15min":1.73},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.087,"15min":1.2410000000000001},"min":{"1min":0.66300000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2450000000000001,"5min":6.6139999999999999,"15min":6.6139999999999999},"last":1.169}]},{"osd":1,"last update":"Mon Sep 16 13:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.94099999999999995,"15min":1.1279999999999999},"min":{"1min":0.55800000000000005,"5min":0.309,"15min":0.309},"max":{"1min":1.0549999999999999,"5min":6.6710000000000003,"15min":6.6710000000000003},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.80200000000000005,"15min":0.80500000000000005},"min":{"1min":0.64000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2969999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.74199999999999999}]},{"osd":2,"last update":"Mon Sep 16 13:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.995,"15min":1.044},"min":{"1min":0.61699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.101,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.095,"15min":1.1870000000000001},"min":{"1min":0.70599999999999996,"5min":0.436,"15min":0.436},"max":{"1min":1.323,"5min":6.2290000000000001,"15min":6.2290000000000001},"last":1.054}]},{"osd":3,"last update":"Mon Sep 16 13:02:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0009999999999999,"15min":1.069},"min":{"1min":0.83899999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.4079999999999999,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.141},{"interface":"front","average":{"1min":1.115,"5min":1.155,"15min":1.2829999999999999},"min":{"1min":0.73699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3540000000000001,"5min":6.3570000000000002,"15min":6.3570000000000002},"last":1.2170000000000001}]},{"osd":4,"last update":"Mon Sep 16 13:01:58 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.86099999999999999}]},{"osd":5,"last update":"Mon Sep 16 13:02:50 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.103,"15min":1.2789999999999999},"min":{"1min":0.65300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2290000000000001,"5min":6.282,"15min":6.282},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.032,"5min":0.94199999999999995,"15min":0.97999999999999998},"min":{"1min":0.71799999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.3260000000000001,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.90200000000000002}]},{"osd":6,"last update":"Mon Sep 16 13:02:35 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.028},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.091}]}]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303544,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068296,"statfs":{"total":96636764160,"available":96325935104,"internally_reserved":0,"allocated":2080768,"data_stored":1051734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 13:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.73099999999999998},"min":{"1min":0.34599999999999997,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.095,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.629},{"interface":"front","average":{"1min":0.753,"5min":0.68600000000000005,"15min":0.67200000000000004},"min":{"1min":0.53600000000000003,"5min":0.254,"15min":0.254},"max":{"1min":1.0940000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.67600000000000005}]},{"osd":2,"last update":"Mon Sep 16 13:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70599999999999996,"15min":0.76900000000000002},"min":{"1min":0.46100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.276,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83299999999999996,"15min":0.878},"min":{"1min":0.5,"5min":0.378,"15min":0.378},"max":{"1min":1.1990000000000001,"5min":3.024,"15min":3.024},"last":0.84199999999999997}]},{"osd":3,"last update":"Mon Sep 16 13:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.93000000000000005,"15min":1.024},"min":{"1min":0.626,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.292,"5min":5.9390000000000001,"15min":5.9390000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82599999999999996,"15min":0.80100000000000005},"min":{"1min":0.51400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.163,"5min":3.544,"15min":3.544},"last":0.88200000000000001}]},{"osd":4,"last update":"Mon Sep 16 13:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.91700000000000004}]},{"osd":5,"last update":"Mon Sep 16 13:02:51 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.1599999999999999,"15min":1.3680000000000001},"min":{"1min":0.76700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.377,"5min":9.4920000000000009,"15min":9.4920000000000009},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.0569999999999999,"15min":1.083},"min":{"1min":0.83899999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.466,"5min":2.5,"15min":2.5},"last":1.0329999999999999}]},{"osd":6,"last update":"Mon Sep 16 13:02:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.153},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0840000000000001}]},{"osd":7,"last update":"Mon Sep 16 13:02:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1279999999999999,"15min":1.1459999999999999},"min":{"1min":0.79600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5149999999999999,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.147,"15min":1.1830000000000001},"min":{"1min":0.80200000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.621,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.2290000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9338,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9108,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21349,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T13:02:56.594 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T13:02:56.913 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T13:02:56.913 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T13:02:56.913 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T13:02:56.913 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:56.980 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T13:02:56.980 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T13:02:56.980 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.059 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T13:02:57.059 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T13:02:57.059 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.120 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T13:02:57.120 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T13:02:57.120 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.190 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T13:02:57.190 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T13:02:57.191 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.276 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T13:02:57.277 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T13:02:57.277 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.357 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T13:02:57.358 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T13:02:57.358 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.437 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T13:02:57.437 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T13:02:57.437 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.503 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T13:02:57.503 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T13:02:57.503 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T13:02:57.503 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.535 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T13:02:57.535 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T13:02:57.535 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.564 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T13:02:57.564 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T13:02:57.565 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T13:02:57.565 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.595 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T13:02:57.596 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T13:02:57.596 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.617 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T13:02:57.617 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T13:02:57.617 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T13:02:57.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-16T13:02:57.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-16T13:02:57.637 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-16T13:02:57.637 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-16T13:02:57.637 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-16T13:02:57.637 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-16T13:02:57.637 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-16T13:02:57.637 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T13:02:57.637 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T13:02:57.638 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-16T13:02:57.660 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T13:02:57.660 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T13:02:57.968 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T13:02:57.968 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T13:02:58.285 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T13:02:58.285 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T13:02:58.491 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T13:02:58.492 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T13:02:58.664 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T13:02:58.664 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T13:02:58.918 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T13:02:58.918 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T13:02:59.108 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T13:02:59.108 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T13:02:59.311 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T13:02:59.311 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T13:02:59.517 INFO:tasks.ceph:Archiving mon data... 2024-09-16T13:02:59.519 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280/data/mon.a.tgz 2024-09-16T13:02:59.519 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-16T13:02:59.526 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.efbEqz3SkY 2024-09-16T13:02:59.526 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.efbEqz3SkY 2024-09-16T13:02:59.631 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.efbEqz3SkY 2024-09-16T13:02:59.651 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.efbEqz3SkY is 420KB 2024-09-16T13:02:59.715 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.efbEqz3SkY 2024-09-16T13:02:59.722 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280/data/mon.c.tgz 2024-09-16T13:02:59.722 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-16T13:02:59.728 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.Ofr63vvONE 2024-09-16T13:02:59.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Ofr63vvONE 2024-09-16T13:02:59.876 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.Ofr63vvONE 2024-09-16T13:02:59.948 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.Ofr63vvONE is 595KB 2024-09-16T13:03:00.011 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.Ofr63vvONE 2024-09-16T13:03:00.018 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280/data/mon.b.tgz 2024-09-16T13:03:00.018 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-09-16T13:03:00.025 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.0Fm5OiqISj 2024-09-16T13:03:00.025 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.0Fm5OiqISj 2024-09-16T13:03:00.172 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.0Fm5OiqISj 2024-09-16T13:03:00.197 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.0Fm5OiqISj is 594KB 2024-09-16T13:03:00.261 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.0Fm5OiqISj 2024-09-16T13:03:00.267 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T13:03:00.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T13:03:00.269 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T13:03:00.286 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-16T13:03:00.286 DEBUG:teuthology.orchestra.run.smithi042:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T13:03:00.288 DEBUG:teuthology.orchestra.run.smithi136:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T13:03:00.305 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T13:03:00.310 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T13:03:00.317 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T13:03:00.318 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/crash to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280/remote/smithi042/crash 2024-09-16T13:03:00.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T13:03:00.370 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/crash to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280/remote/smithi136/crash 2024-09-16T13:03:00.371 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T13:03:00.387 INFO:tasks.ceph:Compressing logs... 2024-09-16T13:03:00.387 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T13:03:00.389 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T13:03:00.409 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29392.log 2024-09-16T13:03:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26174.log 2024-09-16T13:03:00.410 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26965.log 2024-09-16T13:03:00.410 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29392.log.gz 2024-09-16T13:03:00.411 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26831.log 2024-09-16T13:03:00.411 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26174.log.gz 2024-09-16T13:03:00.411 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34756.log 2024-09-16T13:03:00.411 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26965.log.gz 2024-09-16T13:03:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27725.log 2024-09-16T13:03:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26831.log.gz 2024-09-16T13:03:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29647.log 2024-09-16T13:03:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34756.log.gz 2024-09-16T13:03:00.413 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33259.log 2024-09-16T13:03:00.413 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27725.log.gz 2024-09-16T13:03:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36820.log 2024-09-16T13:03:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29647.log.gz 2024-09-16T13:03:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30659.log 2024-09-16T13:03:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33259.log: /var/log/ceph/ceph-client.admin.36820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33259.log.gz 2024-09-16T13:03:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36820.log.gz-5 2024-09-16T13:03:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:--verbose -- /var/log/ceph/ceph-client.admin.28453.log 2024-09-16T13:03:00.415 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30659.log: /var/log/ceph/ceph-client.admin.27777.log 2024-09-16T13:03:00.416 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30659.log.gz 2024-09-16T13:03:00.416 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28821.log 2024-09-16T13:03:00.416 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22303.log 2024-09-16T13:03:00.417 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.35555.log 2024-09-16T13:03:00.417 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28453.log.gz 2024-09-16T13:03:00.417 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33750.log 2024-09-16T13:03:00.417 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27777.log.gz 2024-09-16T13:03:00.418 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28821.log.gz 2024-09-16T13:03:00.418 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35794.log 2024-09-16T13:03:00.421 INFO:teuthology.orchestra.run.smithi136.stderr:gzip /var/log/ceph/ceph-client.admin.22303.log: -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-16T13:03:00.421 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22303.log.gz 2024-09-16T13:03:00.421 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool..log 2024-09-16T13:03:00.422 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/objectstore_tool.35555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22142.log 2024-09-16T13:03:00.422 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/objectstore_tool..log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-16T13:03:00.422 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T13:03:00.422 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.22142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22142.log.gz 2024-09-16T13:03:00.422 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.28017.log 2024-09-16T13:03:00.422 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T13:03:00.422 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/objectstore_tool.28017.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-16T13:03:00.424 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.33750.log: -- /var/log/ceph/ceph-client.admin.33419.log 2024-09-16T13:03:00.424 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33750.log.gz 2024-09-16T13:03:00.424 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27782.log 2024-09-16T13:03:00.424 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35794.log.gz 2024-09-16T13:03:00.424 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30729.log 2024-09-16T13:03:00.424 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33419.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33419.log.gz 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27782.log.gz 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36652.log 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26911.log 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30729.log.gz 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28025.log 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36652.log.gz 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33253.log 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26911.log: /var/log/ceph/ceph-client.admin.28025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28025.log.gz 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose 0.0%-- /var/log/ceph/ceph.tmp-client.admin.17210.log -- replaced with /var/log/ceph/ceph-client.admin.26911.log.gz 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log /var/log/ceph/ceph-client.admin.33253.log: 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33253.log.gz 2024-09-16T13:03:00.425 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37549.log 2024-09-16T13:03:00.426 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.tmp-client.admin.17210.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.17210.log.gz 2024-09-16T13:03:00.426 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36200.log 2024-09-16T13:03:00.426 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37549.log.gz 2024-09-16T13:03:00.426 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.34764.log 2024-09-16T13:03:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36648.log 2024-09-16T13:03:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36200.log.gz 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27524.log 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26938.log 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36648.log.gz 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34801.log 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27524.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27524.log.gz 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35162.log 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26938.log.gz 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34801.log.gz 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37763.log 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35162.log.gz 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32979.log 2024-09-16T13:03:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37763.log.gz 2024-09-16T13:03:00.432 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.6.log: 88.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T13:03:00.432 INFO:teuthology.orchestra.run.smithi136.stderr: 95.3% -- replaced with /var/log/ceph/objectstore_tool.35555.log.gz 2024-09-16T13:03:00.432 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22247.log 2024-09-16T13:03:00.432 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/objectstore_tool.34764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22220.log 2024-09-16T13:03:00.432 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.22247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22247.log.gz 2024-09-16T13:03:00.432 INFO:teuthology.orchestra.run.smithi136.stderr: 95.4% -- replaced with /var/log/ceph/objectstore_tool.28017.log.gz 2024-09-16T13:03:00.437 INFO:teuthology.orchestra.run.smithi136.stderr: 95.5% -- replaced with /var/log/ceph/objectstore_tool.34764.log.gz 2024-09-16T13:03:00.438 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22167.log 2024-09-16T13:03:00.438 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.22220.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.27207.log 2024-09-16T13:03:00.438 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22220.log.gz 2024-09-16T13:03:00.438 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.22167.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.29556.log 2024-09-16T13:03:00.438 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22167.log.gz 2024-09-16T13:03:00.438 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/objectstore_tool.27207.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.36346.log 2024-09-16T13:03:00.438 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22117.log 2024-09-16T13:03:00.440 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36649.log 2024-09-16T13:03:00.440 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32979.log.gz 2024-09-16T13:03:00.440 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27752.log 2024-09-16T13:03:00.440 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.36649.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28774.log 2024-09-16T13:03:00.440 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36649.log.gz 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27752.log.gz 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32754.log 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35437.log 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28774.log.gz 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29324.log 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34659.log 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32754.log.gz 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35437.log.gz 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29324.log.gz 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36616.log 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.27031.log 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26403.log 2024-09-16T13:03:00.441 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36616.log.gz 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.27031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34659.log.gz 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37522.log 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30775.log 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26403.log: 0.0%/var/log/ceph/ceph-client.admin.37522.log: -- replaced with /var/log/ceph/ceph-client.admin.26403.log.gz 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31927.log 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37522.log.gz 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.0.27031.log.gz 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36118.log 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30775.log.gz 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31927.log.gz 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30892.log 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36118.log.gz 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34629.log 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32834.log 2024-09-16T13:03:00.442 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30892.log.gz 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31573.log 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34629.log: 0.0%/var/log/ceph/ceph-client.admin.32834.log: -- replaced with /var/log/ceph/ceph-client.admin.34629.log.gz 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37684.log 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32834.log.gz 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37468.log 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31573.log.gz 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35956.log 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37684.log.gz 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35345.log 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37468.log.gz 2024-09-16T13:03:00.443 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35956.log.gz 2024-09-16T13:03:00.448 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/objectstore_tool.29556.log: /var/log/ceph/objectstore_tool.36346.log: 95.5% -- replaced with /var/log/ceph/objectstore_tool.27207.log.gz 2024-09-16T13:03:00.448 INFO:teuthology.orchestra.run.smithi136.stderr: 95.5%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T13:03:00.448 INFO:teuthology.orchestra.run.smithi136.stderr: -- replaced with /var/log/ceph/objectstore_tool.29556.log.gz 2024-09-16T13:03:00.448 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.22117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22193.log 2024-09-16T13:03:00.448 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22117.log.gz 2024-09-16T13:03:00.448 INFO:teuthology.orchestra.run.smithi136.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.36346.log.gz 2024-09-16T13:03:00.448 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-16T13:03:00.448 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.22193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22193.log.gz 2024-09-16T13:03:00.448 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22275.log 2024-09-16T13:03:00.449 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.4.log: 83.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T13:03:00.449 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.37102.log 2024-09-16T13:03:00.449 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.22275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22275.log.gz 2024-09-16T13:03:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29815.log 2024-09-16T13:03:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28505.log 2024-09-16T13:03:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35345.log: /var/log/ceph/ceph-client.admin.29815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35345.log.gz 2024-09-16T13:03:00.455 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29815.log.gz 2024-09-16T13:03:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30296.log 2024-09-16T13:03:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28628.log 2024-09-16T13:03:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28505.log.gz 2024-09-16T13:03:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27602.log 2024-09-16T13:03:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30296.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30296.log.gz 2024-09-16T13:03:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35929.log 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28628.log.gz 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27042.log 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27602.log.gz 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25968.log 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35929.log: 0.0%/var/log/ceph/ceph-client.admin.27042.log: -- replaced with /var/log/ceph/ceph-client.admin.35929.log.gz 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27042.log.gz 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27908.log 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36184.log 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25968.log.gz 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27908.log.gz 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37711.log 2024-09-16T13:03:00.456 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37198.log 2024-09-16T13:03:00.457 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36184.log.gz 2024-09-16T13:03:00.457 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37711.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37711.log.gz 2024-09-16T13:03:00.457 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25918.log 2024-09-16T13:03:00.457 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29263.log 2024-09-16T13:03:00.457 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37198.log: /var/log/ceph/ceph-client.admin.25918.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37198.log.gz 2024-09-16T13:03:00.457 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25918.log.gz 2024-09-16T13:03:00.457 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36037.log 2024-09-16T13:03:00.457 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29263.log.gz 2024-09-16T13:03:00.457 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32263.log 2024-09-16T13:03:00.457 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36037.log.gz 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30691.log 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32263.log.gz 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31232.log 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31517.log 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30691.log.gz 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36419.log 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31232.log.gz 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30921.log 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31517.log.gz 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36419.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36419.log.gz 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34208.log 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30921.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30921.log.gz 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35821.log 2024-09-16T13:03:00.458 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34208.log.gz 2024-09-16T13:03:00.459 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25891.log 2024-09-16T13:03:00.459 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35821.log.gz 2024-09-16T13:03:00.459 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26396.log 2024-09-16T13:03:00.459 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25891.log.gz 2024-09-16T13:03:00.459 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30973.log 2024-09-16T13:03:00.459 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26396.log.gz 2024-09-16T13:03:00.459 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-16T13:03:00.459 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35039.log 2024-09-16T13:03:00.459 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30973.log.gz 2024-09-16T13:03:00.459 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27311.log 2024-09-16T13:03:00.461 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.28790.log 2024-09-16T13:03:00.461 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/objectstore_tool.37102.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-16T13:03:00.472 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/objectstore_tool.28790.log: 95.2% -- replaced with /var/log/ceph/objectstore_tool.37102.log.gz 95.5% -- replaced with /var/log/ceph/objectstore_tool.28790.log.gz 2024-09-16T13:03:00.472 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-09-16T13:03:00.478 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.7.log: 95.4% -- replaced with /var/log/ceph/objectstore_tool..log.gz 2024-09-16T13:03:00.511 INFO:teuthology.orchestra.run.smithi136.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T13:03:00.514 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35410.log 2024-09-16T13:03:00.514 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35039.log.gz 2024-09-16T13:03:00.514 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36604.log 2024-09-16T13:03:00.514 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27311.log.gz 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35410.log.gz 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26593.log 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36604.log.gz 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25966.log 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26593.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26593.log.gz 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.29590.log 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35767.log 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25966.log.gz 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29045.log 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29590.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29590.log.gz 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32541.log 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35767.log.gz 2024-09-16T13:03:00.515 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36192.log 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29045.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29045.log.gz 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30759.log 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32541.log.gz 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36192.log: /var/log/ceph/ceph-client.admin.27041.log 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36192.log.gz 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35713.log 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30759.log.gz 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26381.log 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27041.log.gz 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31350.log 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35713.log.gz/var/log/ceph/ceph-client.admin.26381.log: 2024-09-16T13:03:00.516 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26381.log.gz 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25964.log 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28291.log 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31350.log: 0.0%/var/log/ceph/ceph-client.admin.25964.log: -- replaced with /var/log/ceph/ceph-client.admin.31350.log.gz 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26727.log 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25964.log.gz 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.28291.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36392.log 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28291.log.gz 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36416.log 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26727.log.gz 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32327.log 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36392.log.gz 2024-09-16T13:03:00.517 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25962.log 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36416.log.gz 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35337.log 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32327.log.gz 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30405.log 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25962.log.gz 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26805.log 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35337.log.gz 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31568.log 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30405.log: /var/log/ceph/ceph-client.admin.26805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26857.log 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30405.log.gz 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26805.log.gz 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26163.log 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31568.log.gz 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35740.log 2024-09-16T13:03:00.518 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26857.log.gz 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29617.log 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37306.log 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26163.log.gz 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35740.log: /var/log/ceph/ceph-client.admin.29617.log: 0.0%/var/log/ceph/ceph-client.admin.37306.log: -- replaced with /var/log/ceph/ceph-client.admin.35740.log.gz 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29617.log.gz 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37306.log.gz 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31709.log 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26180.log 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32658.log 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31709.log: /var/log/ceph/ceph-client.admin.26180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26180.log.gz 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31709.log.gz 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29944.log 2024-09-16T13:03:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31390.log 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32658.log.gz 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37279.log 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29944.log.gz 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29886.log 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37279.log: /var/log/ceph/ceph-client.admin.31390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37279.log.gz 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35004.log 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31390.log.gz 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34155.log 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37117.log 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29886.log.gz 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr: 93.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35004.log: /var/log/ceph/ceph-client.admin.34155.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34155.log.gz 2024-09-16T13:03:00.520 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35004.log.gz 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37630.log 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35107.log 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37117.log.gz 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35983.log 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37630.log.gz 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.35107.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27258.log 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35107.log.gz 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37360.log 0.0% 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35983.log.gz 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35464.log 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33867.log 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37360.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27258.log.gz 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37360.log.gz 2024-09-16T13:03:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35464.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35464.log.gz-5 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr:--verbose -- /var/log/ceph/ceph-client.admin.36145.log 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 /var/log/ceph/ceph-client.admin.33867.log: --verbose -- /var/log/ceph/ceph-client.admin.37144.log 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33867.log.gz 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36414.log 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36145.log.gz 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31267.log 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37144.log.gz 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27397.log 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33343.log 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36414.log.gz 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31267.log.gz 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33128.log 2024-09-16T13:03:00.522 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27397.log.gz 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27488.log 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33343.log.gz 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30073.log 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34375.log 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33128.log.gz 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27488.log.gz 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36198.log 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30073.log.gz 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26620.log 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34375.log.gz 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34559.log 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36198.log.gz 2024-09-16T13:03:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26779.log 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26620.log.gz 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26181.log 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34559.log.gz 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29989.log 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26779.log.gz 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31671.log 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26181.log.gz 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26164.log 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29989.log.gz 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33189.log 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31671.log.gz 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29196.log 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26164.log.gz 2024-09-16T13:03:00.524 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37603.log 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33189.log.gz 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36888.log 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29196.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29196.log.gz 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32881.log 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37603.log.gz 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37225.log 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36888.log.gz 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27950.log 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32881.log.gz 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32433.log 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37225.log.gz 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27950.log.gz 2024-09-16T13:03:00.525 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25973.log 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25818.log 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32433.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32433.log.gz 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25973.log.gz 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25864.log 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37441.log 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33394.log 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25818.log.gz 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25864.log.gz 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26673.log 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37441.log.gz 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27043.log 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33394.log.gz 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35224.log 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37036.log 2024-09-16T13:03:00.526 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26673.log.gz 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27043.log.gz 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37387.log 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35224.log.gz 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26405.log 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37036.log.gz 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37252.log 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37387.log.gz 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37414.log 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26405.log.gz 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37333.log 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37252.log.gz 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31100.log 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37414.log.gz 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36870.log 2024-09-16T13:03:00.527 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37333.log.gz 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.31100.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33551.log 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31100.log.gz 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27515.log 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36870.log.gz 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37576.log 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33551.log.gz 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33604.log 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27515.log.gz 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36393.log 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37576.log.gz 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34234.log 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33604.log.gz 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T13:03:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36393.log.gz 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36893.log 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34234.log.gz 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36886.log 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32009.log 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36893.log.gz 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28978.log 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36886.log.gz 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28295.log 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32009.log.gz 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36824.log 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28978.log.gz 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32429.log 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28295.log.gz 2024-09-16T13:03:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34502.log 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36824.log.gz 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34335.log 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32429.log.gz 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29443.log 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34502.log.gz 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33344.log 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34335.log.gz 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36064.log 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29443.log.gz 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37090.log 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33344.log.gz 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37657.log 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36064.log.gz 2024-09-16T13:03:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36199.log 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37090.log.gz 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30623.log 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37657.log.gz 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35287.log 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36199.log.gz 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26997.log 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30623.log.gz 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32723.log 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35287.log.gz 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.26997.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26406.log 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26997.log.gz 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37495.log 2024-09-16T13:03:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32723.log.gz 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35517.log 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26406.log.gz 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32930.log 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37495.log.gz 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33040.log 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35517.log.gz 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32181.log 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32930.log.gz 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22183.log 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33040.log.gz 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30496.log 2024-09-16T13:03:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32181.log.gz 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.22183.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34883.log 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22183.log.gz 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 /var/log/ceph/ceph-client.admin.30496.log: --verbose -- /var/log/ceph/ceph-client.admin.28710.log 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30496.log.gz 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27698.log 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34883.log.gz 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28501.log 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28710.log.gz 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.27698.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32121.log 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27698.log.gz 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28501.log: /var/log/ceph/ceph-client.admin.29773.log 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28501.log.gz 2024-09-16T13:03:00.533 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30533.log 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32121.log.gz 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33630.log /var/log/ceph/ceph-client.admin.29773.log: 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29773.log.gz 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.30533.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31050.log 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30533.log.gz 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26188.log 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33630.log.gz 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30249.log 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31050.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31050.log.gz 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29148.log 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26188.log.gz 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35544.log 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30249.log.gz 2024-09-16T13:03:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25729.log 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29148.log.gz 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32222.log 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35544.log.gz 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26647.log 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25729.log.gz 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32222.log: /var/log/ceph/ceph-client.admin.26884.log 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32222.log.gz 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36609.log 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26647.log.gz 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36654.log 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26884.log.gz 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26394.log 2024-09-16T13:03:00.535 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36609.log.gz 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.36654.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26701.log 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36654.log.gz 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31800.log 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26394.log.gz 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33455.log 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26701.log.gz 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35077.log 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31800.log.gz 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32596.log 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33455.log.gz 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29085.log 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35077.log.gz 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33087.log 2024-09-16T13:03:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32596.log.gz 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30591.log 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29085.log.gz 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33087.log: /var/log/ceph/ceph-client.admin.35468.log 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33087.log.gz 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 /var/log/ceph/ceph-client.admin.30591.log: --verbose -- /var/log/ceph/ceph-client.admin.33780.log 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30591.log.gz 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35468.log: /var/log/ceph/ceph-client.admin.26753.log 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35468.log.gz 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28666.log 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33780.log.gz 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35686.log 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26753.log.gz 2024-09-16T13:03:00.537 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28666.log: /var/log/ceph/ceph-client.admin.36195.log 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28666.log.gz 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.35686.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35625.log 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35686.log.gz 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25792.log 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36195.log.gz 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25972.log 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35625.log.gz 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37063.log 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25792.log.gz 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32271.log 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25972.log.gz 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25969.log 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37063.log.gz 2024-09-16T13:03:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36829.log 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32271.log.gz 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33708.log 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25969.log.gz 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.36829.log: -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36829.log.gz 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32083.log 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33708.log.gz 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27306.log 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34446.log 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32083.log.gz 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:gzip /var/log/ceph/ceph-client.admin.27306.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37738.log 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27306.log.gz 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34446.log: gzip -5 0.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.34446.log.gz 2024-09-16T13:03:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35313.log 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28085.log 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37738.log.gz 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31825.log 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35313.log.gz 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36091.log 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr: 91.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28085.log.gz 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35848.log 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31825.log.gz 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36091.log.gz 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35108.log 2024-09-16T13:03:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35848.log.gz 2024-09-16T13:03:00.541 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31967.log 2024-09-16T13:03:00.541 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29551.log 2024-09-16T13:03:00.541 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35108.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35108.log.gz 2024-09-16T13:03:00.557 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36835.log 2024-09-16T13:03:00.557 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31967.log.gz 2024-09-16T13:03:00.557 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36404.log 2024-09-16T13:03:00.557 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29551.log.gz 2024-09-16T13:03:00.557 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30723.log 2024-09-16T13:03:00.557 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36835.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35576.log 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36835.log.gz 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36404.log.gz 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34080.log 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30723.log.gz 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30207.log 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35576.log.gz 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35902.log 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34080.log.gz 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33124.log 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30207.log.gz 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33504.log 2024-09-16T13:03:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35902.log.gz 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28899.log 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33124.log.gz 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28155.log 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33504.log.gz 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27445.log 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28899.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28899.log.gz 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28155.log.gz 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26377.log 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27190.log 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27445.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27445.log.gz 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26377.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.26377.log.gz-5 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:--verbose -- /var/log/ceph/ceph-client.admin.36396.log 2024-09-16T13:03:00.559 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33968.log 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27190.log.gz 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28343.log 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36396.log.gz 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36010.log 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33968.log.gz 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32882.log 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28343.log.gz 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36010.log.gz 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30847.log 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26172.log 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32882.log.gz 2024-09-16T13:03:00.560 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35654.log 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30847.log.gz 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30373.log 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26172.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.31472.log -- replaced with /var/log/ceph/ceph-client.admin.26172.log.gz 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35654.log: gzip 0.0%/var/log/ceph/ceph-client.admin.30373.log: -- replaced with /var/log/ceph/ceph-client.admin.35654.log.gz-5 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:--verbose -- /var/log/ceph/ceph-client.admin.29648.log 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30373.log.gz 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22320.log 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31472.log.gz 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28081.log 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29648.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29648.log.gz 2024-09-16T13:03:00.561 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.22320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36613.log 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28081.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28081.log.gz 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.22320.log.gz 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28194.log 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27645.log 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25970.log 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36613.log.gz/var/log/ceph/ceph-client.admin.28194.log: 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28194.log.gz 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27645.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26395.log 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27233.log 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25970.log.gz 2024-09-16T13:03:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26395.log.gz 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25844.log 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.27233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27233.log.gz 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36194.log 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35575.log 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.25844.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25844.log.gz 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36194.log.gz 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36188.log 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22020.log 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35575.log.gz 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29494.log 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.22020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28509.log 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22020.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36188.log.gz 2024-09-16T13:03:00.563 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29494.log.gz 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28509.log.gz 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34054.log 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28259.log 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28936.log 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37171.log: /var/log/ceph/ceph-client.admin.34054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34054.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28259.log.gz 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35875.log 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30099.log 2024-09-16T13:03:00.564 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28936.log.gz 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36403.log 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33927.log 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.30099.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35875.log.gz 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30099.log.gz 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36403.log.gz 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31163.log 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33927.log.gz 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28384.log 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31163.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26189.log 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31163.log.gz 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28384.log.gz 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34927.log 2024-09-16T13:03:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28597.log 2024-09-16T13:03:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26189.log: /var/log/ceph/ceph-client.admin.34927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34927.log.gz 2024-09-16T13:03:00.566 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26189.log.gz 2024-09-16T13:03:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T13:03:00.566 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.28597.log: /var/log/ceph/ceph-osd.2.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28597.log.gz 2024-09-16T13:03:00.587 INFO:teuthology.orchestra.run.smithi136.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T13:03:00.651 INFO:teuthology.orchestra.run.smithi042.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T13:03:00.710 INFO:teuthology.orchestra.run.smithi042.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T13:03:00.935 INFO:teuthology.orchestra.run.smithi136.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-16T13:03:01.695 INFO:teuthology.orchestra.run.smithi042.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T13:03:02.586 INFO:teuthology.orchestra.run.smithi136.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-16T13:03:02.638 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-16T13:03:02.638 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-16T13:03:02.638 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-16T13:03:02.638 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-16T13:03:02.639 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-16T13:03:02.639 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-16T13:03:02.639 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-16T13:03:03.341 INFO:teuthology.orchestra.run.smithi136.stderr: 96.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-16T13:03:03.591 INFO:teuthology.orchestra.run.smithi042.stderr: 96.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T13:03:04.052 INFO:teuthology.orchestra.run.smithi136.stderr: 96.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-16T13:03:04.053 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-09-16T13:03:04.053 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m3.658s 2024-09-16T13:03:04.053 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m9.275s 2024-09-16T13:03:04.053 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m0.312s 2024-09-16T13:03:05.920 INFO:teuthology.orchestra.run.smithi042.stderr: 96.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T13:03:09.075 INFO:teuthology.orchestra.run.smithi042.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-16T13:03:09.076 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-16T13:03:09.076 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m8.684s 2024-09-16T13:03:09.076 INFO:teuthology.orchestra.run.smithi042.stderr:user 0m18.262s 2024-09-16T13:03:09.076 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m0.747s 2024-09-16T13:03:09.077 INFO:tasks.ceph:Archiving logs... 2024-09-16T13:03:09.077 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280/remote/smithi042/log 2024-09-16T13:03:09.078 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T13:03:11.359 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280/remote/smithi136/log 2024-09-16T13:03:11.360 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T13:03:12.493 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T13:03:12.502 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T13:03:12.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T13:03:12.504 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T13:03:12.587 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T13:03:12.596 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T13:03:12.596 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T13:03:12.598 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T13:03:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T13:03:12.617 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================== 2024-09-16T13:03:12.617 INFO:teuthology.orchestra.run.smithi042.stdout:+hv01.front.sepi 67.205.162.81 3 u 38 64 177 0.133 1.258 1.089 2024-09-16T13:03:12.617 INFO:teuthology.orchestra.run.smithi042.stdout:+hv02.front.sepi 63.231.80.2 3 u 40 64 177 0.132 0.231 0.972 2024-09-16T13:03:12.617 INFO:teuthology.orchestra.run.smithi042.stdout:*hv03.front.sepi 74.6.168.72 3 u 46 64 177 0.144 0.789 1.230 2024-09-16T13:03:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T13:03:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T13:03:12.621 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2024-09-16T13:03:12.621 INFO:teuthology.orchestra.run.smithi136.stdout:+hv01.front.sepi 67.205.162.81 3 u 44 64 177 0.078 0.157 0.396 2024-09-16T13:03:12.621 INFO:teuthology.orchestra.run.smithi136.stdout:+hv02.front.sepi 63.231.80.2 3 u 47 64 177 0.068 -0.879 0.147 2024-09-16T13:03:12.622 INFO:teuthology.orchestra.run.smithi136.stdout:*hv03.front.sepi 74.6.168.72 3 u 42 64 177 0.065 -0.162 0.225 2024-09-16T13:03:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T13:03:12.622 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T13:03:12.631 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T13:03:12.632 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T13:03:12.640 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T13:03:12.650 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T13:03:12.658 INFO:teuthology.task.internal:Duration was 785.363060 seconds 2024-09-16T13:03:12.658 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T13:03:12.667 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T13:03:12.667 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T13:03:12.669 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T13:03:12.721 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T13:03:12.722 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T13:03:12.722 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T13:03:12.733 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-09-16T13:03:12.734 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T13:03:12.746 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T13:03:12.747 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T13:03:12.749 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T13:03:12.790 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T13:03:12.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T13:03:12.835 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T13:03:12.922 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T13:03:12.932 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T13:03:12.933 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T13:03:12.949 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T13:03:12.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T13:03:12.978 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T13:03:12.980 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T13:03:12.993 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-09-16T13:03:12.994 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-09-16T13:03:13.034 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T13:03:13.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T13:03:13.066 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T13:03:13.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T13:03:13.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T13:03:13.089 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T13:03:13.090 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280/remote/smithi042 2024-09-16T13:03:13.090 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T13:03:13.121 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907280/remote/smithi136 2024-09-16T13:03:13.122 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T13:03:13.147 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T13:03:13.147 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T13:03:13.150 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T13:03:13.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T13:03:13.164 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T13:03:13.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T13:03:13.173 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T13:03:13.174 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T13:03:13.199 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T13:03:13.204 INFO:teuthology.orchestra.run.smithi042.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 13:03 /home/ubuntu/cephtest 2024-09-16T13:03:13.207 INFO:teuthology.orchestra.run.smithi136.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 13:03 /home/ubuntu/cephtest 2024-09-16T13:03:13.209 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T13:03:13.218 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T13:03:13.238 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 785.3630595207214 flavor: default owner: scheduled_sangadi@teuthology success: true 2024-09-16T13:03:13.238 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T13:03:13.316 INFO:teuthology.run:pass